Completed features of the Hildon Application Manager

====================================================

See FUTURE for planned features.

* Use HildonStackableWindow instead of a single HildonWindow. [dkedves]

* scare_user_with_legalese () notice should appear on

  ip_install_with_info (). [vjaquez]

Instead of two sequential dialogs, join them in only one.

* Make a h-a-m/domains config directory. [vjaquez]

For dropping domain.xexp fragments, just like h-a-m/catalogues.  Also

see the lower priority task "config-cleanup".

* Drop support for memory cards completely [vjaquez]

Everything related to installation from memory cards should be

removed.

* Update the notification plugin for the new hildon status menu in

  hildon-desktop [vjaquez]

* Remove the free space verification from the UI and move it to the

  apt-worker [vjaquez]

* Avoiding invisible interaction flows

Currently, it is possible to start an interaction flow (using

start_interaction_flow, say) without immediatly popping up a dialog

that would represent this interaction flow.

The delays used to be shortish, but they can now be quite long because

of big OS updates.

For example, opening the "Details" dialog for a "OS2008 Reloaded"

update can take a couple of seconds and there is therefore a long gap

between activating the button and the dialog opening.

Two things need to be done: the preparing of large operations needs to

be sped up, and potential gaps in the UI flow need to be plugged.

This feature is about pluggin the gaps in the UI.

First, the gaps need to be identified: look for uses of

start_interaction_flow etc and make sure that no operation is

performed between calling this start_interaction_flow function and

showing the first dialog.

No apt-worker request can be assumed to be quick (not even things like

APTCMD_GET_CATALOGUES that don't require significant processing) since

the apt-worker might be busy with something else before being able to

process a quick request.

If information from the apt-worker is needed to construct the dialog,

the code could be changed to show an 'empty' dialog with insensitive

buttons and a "updating" progress banner.  When the information

arrives, the dialog should be filled and the buttons should be made

sensitive.

Please prepare a list of these cases so that we can quickly see

whether this approach needs to be refined in some cases.

The details dialog is already a bit special: do not make the "Close"

button insensitive while waiting for more information from the

apt-worker.  It should be possible to close the dialog at any time.

The apt-worker request does not need to be canceled in that case, but

the eventual reply should be safely ignored, of course.

Maybe some apt-worker requests need to split in two so that the first

half can finish quickly.

* Get rid of the temporary apt state concept [victor]

When executing a 'card-install' interaction flow, the apt-worker uses

a complete, separate, and temporary state that only includes the

repository on the card.  This feature is responsible for a lot of

code complexity.

The reason we have the secondary state is that we do not want to

download the network repository indices again and do not want to

recompute the caches.

However, card-installs should be very, very rare and we do not need to

optimize for them.

Thus, let's make the code simpler at the expense of card-installs.

First, the card-install operation will be re-defined to access the

network repositories during the operation.  Dependencies can be

fulfilled from them.

Then, the card-install will simply put the card repository to

/etc/apt/sources.list.d/hildon-application-manager-temp.list, and

remove that file afterward (via a new apt-worker command, probably).

For extra robustness, this file should be removed by the apt-worker

when it starts.

Finally, the whole temporary state machinery can be removed.

* Clean up pre-defined catalogues mechanics [victor]

The pre-defined catalogues of the Application manager should be

handled according to the new packaging policy (doc/packpol-conf.txt).

** Spec

Roughly, this means that we need a strict division into catalogue

entries that are completely 'owned' by packages, and catalogue entries

that are competely owned by the user.

All the pre-defined catalogues should be owned by packages and can not

be edited by the user.  They are not backed up.

However, the enabled/disabled flag for each catalogue entry is owned

by the user: the user can still disable a pre-defined catalogue.  This

flag is backed up.

** Implementation hints

Right now, all catalogue information is stored in a single file,

/etc/h-a-m/catalogues.  This should be changed so that only user

catalogues are stored there.  Packages should be able to drop files

into /usr/share/hildon-application-manager/catalogues/.

When reading the catalogue information, the xexps from

/etc/h-a-m/catalogues and /usr/share/h-a-m/catalogues/ should be

merged.  Concretely, the /etc/h-a-m/catalogues file contains the

enabled/disabled flags for the catalogues in

/usr/share/h-a-m/catalogues directory.  For this, packaged catalogue

entries need to be identified reliably: a new "id" attribute should be

used for this.

It is probably a good idea to remember where a catalogue has been read

from: after reading its xexp from a file

/usr/share/h-a-m/catalogues/foo, we add a <file>foo</file> element to

the xexp.  Then catalogues are identified by their "file" plus "id"

attributes, making it easier to choose unique ids.

User catalogues have no "file" attributes.

When saving the catalogues after the user has edited them, only the

/etc/h-a-m/catalogues file should be touched.  That is, the xexps for

a catalogue without a "file" attribute are written to that file, for

other xexps, only the "file", "id" and "disabled" attribute are

written.

These rules should also be followed by the

hildon-application-manager-config utility.  Changes made with that

tool count as user edits.

* Get rid of the "essential"/readonly repositories [victor]

The Application Catalogue UI dialog has a heavy use of readonly

repositories which are based on the repositories marked as

"essential", but with the new package catalogue mechanism, this

feature is deprecated.

The readonly flag must be activaded only when is a package catalogue.

* Get rid of file system synchronicity assumption [victor]

Some code in the Application manager might rely on the synchronous

nature of JFFS2.  This will be a problem should we ever move to some

other file system that hasn't this nature, like ext3 or UBIFS.

See

    http://www.linux-mtd.infradead.org/doc/ubifs.html#L_writeback

The code needs to be reviewed and appropriate fsync calls (etc) should

be inserted.

The xexp_write function is a good candidate since it already makes

some atomicity guarantees (and its documentation should be updated to

mention the syncronicity).

* Show repository that a package is going to be installed from

  [victor, vjaquez-show-repo]

The "Details" dialog should be extended somehow to show the

repositories / catalogues that a package is going to be installed

from.

* Remove UI from maemo-select-menu-location [mvo, mvo-remove-menu-location].

It's annoying.

* Repairing broken packages [mario]

If a visible package is in a broken state, it should also be shown in

the "Check for Updates" view and updating it should try to unbreak the

package.

Just using mark_for_install might be good enough for unbreaking a

package.

Right now, packages that need to be reinstalled are shown in the

"Installable applications" view.  This shouldn't be done anymore.

Open issue: what about broken invisible packages?  Their brokenness

could be charged to the visible parents, but that will charge a broken

libgtk, say, on all visible packages.

* Ignoring packages that violate domain boundaries [mvo]

The Application manager remembers for each package the 'domain' where

it was installed from originally.  Updates to a package that come from

another domain are rejected.

Right now, the Application manager fails installations that have one

or more of these domain violations.  Instead, it should just ignore

packages from a wrong domain.  This will get rid of the most hard core

error message of the AM.

There is already a hook into libapt-pkg to deal with the domain

system.  This hook might be good enough, it might have to be changed.

Some thoughts should be spent on whether to move the whole domain

concept into libapt-pkg.  Maybe writing a custom DepCache::Policy is

the way to go.

* Speeding up preparation of large operations [mvo]

Computing an update (calling mark_install in the apt-worker, etc) used

to be quick enough, but now it can be quite slow since we have large

OS updates.  This needs to be sped up.

When performing an installation, the frontend asks the backend to

compute the same update three times or so.  Either the frontend needs

to ask less often or the backend should keep the last operation in the

cache.

* Automatic recovery after an interrupted operation [mvo]

When a package management operation is interrupted abruptly, the

system might still be able to recover by trying to complete the update

on the next boot.

To acomplish this, the apt-worker is run in a new "rescue" mode on

every boot, as early as possible.

The rescue mode will determine whether any repairs are needed, and if

so, will attempt them.  Determining whether any repairs are needed

must be fast.

Steps:

- Move flash-and-reboot functionality into apt-worker so that it is

  available during repairs.

- Have the apt-worker write a journal for each INSTALL_PACKAGE

  operation.  The journal entry should contain the package name and

  the location with the packages.

- Add the "rescue" command line command which will look at the

  journal.  If the journal is non-empty, try to finish it.

Finishing the journal means first running "dpkg --configure -a",

ignoring any errors, and then redoing the installation described in

the journal.  Then, "dpkg --configure -a --force-all" is run, for good

measure.

If any packages would need to be downloaded, the repair is not done.

If the required, the device is rebooted, or flash-and-reboot is run at

the end, or booting is allowed to continue.

UI of this to be decided.

* Pushing of messages into the update notifier plugin [felipe]

Product management wants to be able to notify the user about

significant new applications that are available for theri internet

tablets.  They want to use the existing update notifier statusbar

plugin for that.

The update-notifier plugin should be able to watch a given URI and

present information from this URL via the statusbar icon.  When the

contents of the resource behind the URI has changed, the icon should

start blinking; when the user taps it, the blinking should stop and a

menu with information from the resource should open; when the user

selects an action from the menu, the icon should disappear.

The original update notification functionality of the statusbar plugin

should have priority: the resource watching functionality should only

control the icon when the update notification functionality would hide

the icon.

The URI should be checked at the same time as the checking for

updates.

Final details about the UI and the content of the watched resource are

upcoming.  The menu should have two action items in it: "Tell me

more", which opens the web browser with a given URI, and "No thanks",

which does nothing else than hide the icon.

To get started, the content of the resource should be assumed to be a

xexp of the form:

  <info>

   <text>Blah, blah</text>

   <uri>http://www.example.com/</uri>

  </info>

This should yield a menu like this:

  Blah, blah                <- insensitive

  -----------

  Show me more              <- open browser on http://www.example.com

  No, thanks.               <- hides icon

It is important that checking the URI does not block the process.  If

needed, a separate process should be spawned (no threads, please).

* Automatic checking for updates in the background

- APTCMD_UPDATE_PACKAGE_CACHE is renamed to APTCMD_CHECK_UPDATES and

  made cancellable at every point, not just during downloading.  Also,

  it should not report errors at the end of the operation but store

  them away for later retrieval.  It should also collect the

  information necessary for the notification plugin (number of

  available updates, classified into "OS", "Nokia", and "Other") and

  store it away.

- The apt-worker should be able to be invoked from the command-line to

  perform the APTCMD_CHECK_UPDATES operation.  When invoked in this

  way, the operation should be cancellable (using SIGINT, say).  Also,

  it should refuse to run when the apt-worker is already running as

  the backend for the AM.

- When the apt-worker starts in its usual role as the backend for the

  Application Manager, it should cancel already running instances of

  itself that have been started from the command line.

- The Application Manager should have the new "Update All" button in

  the "Check for Updates" view.

- There is a new component: the notification statusbar plugin.  The

  notification plugin runs always and is in one of three different

  states: invisible/visible/blinking.

- The notification plugin invokes the apt-worker in comand-line mode

  to perform APTCMD_CHECK_UPDATES 'occasionally'.  Such a operation is

  attempted when the following conditions are all true:

  - The last APTCMD_CHECK_UPDATES has stopped (successfully or not,

    interrupted or not) 24 hours or more ago.

  - There is a active WLAN connection (no Bluetooth).

  When the APTCMD_CHECK_UPDATES can not be performed because the

  apt-worker is already running as the backend for the Application

  Manager, the notification plugin should send a message to the AM

  that requests it to do the operation instead.

- When the Application Manager receives such a request, it will

  perform it as early as possible.  It is possible to do it when the

  AM has been idle for at least one minute.  Idle means: no

  interaction flow is active.

- The state changes of the plugin are as follows:

  - when the user taps it (to open the menu), it stops blinking.

  - when a APTCMD_CHECK_UPDATES initiated by the plugin itself was

    sucessful and updates are available and the plugin was invisible

    before, it starts blinking.

  - when the "Check for updates" view in the Application Manager is

    opened, or when the user invokes the Application Manager from the

    plugin menu and the "Check for Updates" view is already open, the

    plugin becomes invisible.

- The notification plugin should store its visibility state

  permanently across reboots.

- The notification plugin should be notified by apt-worker whenever

  the information for it changes, and the plugin should then read that

  information and reconstruct its menu.