org.melati Internals Description (document $Revision: 1.1 $)

# `org.melati` Internals Description (document $Revision: 1.1 $)

This document provides a description of anything noteworthy in the workings of the system. See also the system's master QA document.

## Foundations

### Global data structure and format

The fundamental data structures used in the system are:

... data dictionary; content management of static web pages; admin configuration

### Data access

The interface through which the code accesses the data is:

...

### Pervasive libraries

The libraries that underpin large parts of the project are:

... the litmus test is ``Will I have to change vast amounts of other code if I change this?''

### Pervasive graphic designs

The graphic designs that have wide implications for the layout of the system's interfaces are:

...

## Sub-parts

The parts into which the system naturally falls are:

... minimise the ``surface'' area of interface; make each piece doable by at most two developers; describe the interface it offers and the other parts it relies on; include graphics designs

### Data structure definition processor

#### Parser

The language is simple. The only wrinkle is that the definitions are recursive. So we use a simple custom top-down parser (with, obviously, Java's library tokenizer).

#### Base class generation

The auto-generated base classes for the business objects wrapping the table rows are quite simple, because most of the logic of fetching and interpreting table rows sits in the `Table` itself (including invalidation of cached rows when the schema is changed).

It probably doesn't matter if the programmer is allowed to provide their own `getCanRead`, `getCanWrite` methods?

#### Database structure generation

Some more auto-generated code needs to describe the minimum database scheme the app expects to find, including indexes.

### Transactions and cacheing

Transactions and cacheing are handled at the level of the raw data, underneath the access control and object logic layer (for which see below).

### Access control and object logic

Every object handle is authenticated for a given user.

### Runtime of the persistent store

#### Representation

Each table row is meant to be represented by a single `Record` object; contextual information which can affect the results of accesses to its fields, such as the user's access token and the transaction relative to which she is working, are encoded in the running thread.

The actual data can very between different concurrent transactions if one is uncommitted. Each version could be stored most cheaply as an `Object[]`, but we choose instead to wrap them up in a `Row` object for the reasons explained in `Row.java`.

#### Non-uniqueness of `Record`s

However, it's not quite quite possible in Java 1.1 to guarantee that a row won't sometimes get two `Record` handles. After the following sequence of events ...

- some application code c asks for a certain row
- the row is loaded into the cache and a `Record` r created for it
- r is passed back to c
- r is deleted from the cache because of some other activity
- c asks for the same row again
- because the row is now absent from the cache again, it is loaded once more and a new `Record` r' is created for it
- r' is passed back to c

... c has two different references to the same object. Of this situation is going to be uncommon, but the only way to avoid it would be to use weak pointers (oh, alright then, ``SoftReference`s') to allow objects to be marked as purgeable from the cache but still stay alive until there were known to be no references to them left.

#### The cache

What happens if the administrator changes the database while a write is being prepared? It needs to fail. E.g. enumeration of generic fields can produce odd results in this case.

We know everything that can possibly have changed in a session, since we forbid SQL `UPDATE`s which are not mediated through setter methods. So everything that comes in from the database and is not already in the session cache can go straight into the committed cache. NB---just bear in mind that new `Row`s mustn't exist in the committed cache until the session in which they are created is committed.

#### Access checks

Hmmm, it's necessary that the programmer's overriding `assertReadable` / `assertWriteable` should be run with maximal permissions. Alternatively, we could provide `protected` check-free versions of the getter methods---but then the programmer's utility methods would have to come in check-free versions as well.

- application code calls `getObject("foo")`
- the name is mapped: `getObject(7)`
- the number is mapped: `getFoo()`
- the programmer's one calls `super.getFoo()`
- it calls: `_getObject(7)`
  - that calls: `assertReadable()`
  - which determines the access token
  - and calls `assertReadable(token)` as root
  - the programmer's one calls `super.assertReadable(token)`
  - it determines the `canRead` capability (which doesn't recursively trigger access checks because it's running as root)
  - and checks that `accessToken` carries `canRead`
- and then `rowUnprotected`
- which retrieves the row from the session or the committed cache
- field 7 is extracted and converted

This all begins to put the potential inefficiency of `token.givesCapability(canRead)` into perspective.

There is the option, for the future, of providing a `handle' which doesn't have to do an access check for every single `read'. Or, we go back to the idea of providing `getFoo_opaque`, `getFoo_write` and `getFoo` (for read access). Can we alternatively cache one or two access tokens which are known to have the appropriate capabilities?

Alternative: specialise every handle you pass out specialised to a particular `AccessToken`. Mark it with the permissions (on first need?). Check that the thread is carrying that token, and that it hasn't been revoked (if indeed that's possible), but no more is required. Have the handle carry its own values (when they're changed), and add a `commit` and an `update`. Consequences:

- more garbage collection
- less checking (in fact very little)
- no problem with batching writes, since that's done explicitly---ACTUALLY we can put off all writes until a (non-committed) session is committed if we like?
- loss of equality testing/simple `synchroniz`ation (might be sensible to check for that, actually, and warn---easy at runtime using reflection)

Perhaps we offer this as a possibility? If people want to do a number of batched writes, or they want to read a lot of fields from a snapshot of a record, they say

>     // Whatever fields the template reads (from the invoice as such),
>     // no further (expensive) access or session checks will be done.

>     template.context.put("invoice", invoice.snapshotRead());

or

>     InvoiceSnapshot invoiceSnapshot = invoice.snapshotReadWrite();
>     invoiceSnapshot.setTaxDate(inputs.get("taxDate"));
>     invoiceSnapshot.setReceiver(inputs.get("receiver"));
>     invoiceSnapshot.fixate();

One problem is that any method on a programmer-subclassed object can have unpredictable side effects, including for instance reads and/or writes on linked objects. This means that a snapshot `set' method may have an effect on non-snapshots, unless we say that snapshot objects call up their own snapshots when links are followed. Perhaps it's better to say: we simply don't write down changes made in a non-committing session unless asked. We simply leave all the changes in the cache until commit time. But it's still nice to use snapshot objects to avoid doing endless access checks.

The master `Record` caches the session information. The slave `Record`s cache the authentication information for a single `AccessToken`.

#### Subsets

Need to be able to: select themselves out of the db with a prepared statement; trigger to be changed when a field of a record is changed.

### Admin system

#### How does `Admin.java` work currently?

`Action' | Incoming `.wm` | Function | Outgoing `.wm`
--- | --- | --- | ---
`tables` | various | list tables in database | `AdministrationTables1`
`select` | various | ``Get all relevant drop boxes to SELECT a set of records and perhaps LIST them or LISTEDIT them'' [yeah right, as you do]. I think this means: the screen where you choose the criteria by which you want a report of records from a table. | `AdministrationEditList1`, `AdministrationList1`
`briefselect` | `select_frameset2.wm` | ``Get one drop boxes to BRIEFSELECT a set of records (the "first" select box)''. I have no idea what this does. | `select2.wm`
`display` | none, apparently | ``DISPLAY one record.'' Dead simple. | `AdministrationDisplay1`
`edit` | various | ``EDIT one record.'' I don't understand `UploadURL`. | `AdministrationEdit1`; `AdministrationEditPageContents1`
`update` | `AdministrationEdit1`; `AdministrationEditPageContents1`; `edit2`; `editadd2` | ``UPDATE one record.'' Handles incoming add/edit/delete. | `AdministrationUpdate1`
`update multiple` when not `add_button` | ``AdministrationEdit1.wm`; `editlist2.wm`` | ``UPDATE MULTIPLE records.'' Apparently handles incoming `add/edit/delete all'. Don't understand. | `AdministrationMultipleUpdate1`
`update multiple` when `add_button` | ``AdministrationEdit1.wm`; `editlist2.wm`` | ``ADD a new record to a MULTIPLE records list.'' Apparently handles incoming `add'. Don't understand. | `AdministrationEditList1`

## Important issues

### Performance

The system meets its performance requirements because:

... refer to the requirements analysis

### Security

The system is secure because:

... refer to the requirements analysis

### Robustness

The system is robust because:

... refer to the requirements analysis

### Stability

The system is stable because:

... refer to the requirements analysis

## Risk analysis

The obvious ways in which this scheme might be flawed are:

...

## About this document

### Authors

William Chesters <williamc@paneris.org>

Most recent CVS $Author: timp $ @paneris.org

### Quality

The current quality level of this document (including any computer-generated documentation, such as javadoc, which belongs with it) is: Personal.

These are simply my scrappy working notes. I justify this by saying that it's only me developing the core code, and that will be its own best documentation (along with a post hoc description of the mechanisms).

### Readership and purpose

- The project leader should feel happy about how the system will be implemented.
- The developers should know as far as possible what they are going to do.
- Future maintainers should be able to understand any non-obvious details about how the system works.

### History

The important points in the life of this document are listed below (for everyday update information, consult its CVS log).

Date | Event
--- | ---
(not yet) | Approved at release quality level by ...

$Log: Internals.html,v $
Revision 1.1 2005/11/21 22:01:49 timp
Moved from site/doc
Revision 1.9 2000/02/29 09:53:02 williamc
Finish recovering from disaster; point out in the docs that you can 'add methods to table rows'
Revision 1.1 2000/02/08 15:45:08 williamc
Initial revision
Revision 1.2 2000/02/04 18:28:34 williamc
Add QA stub; explain DSD-admin interactino a little better