Albatross - a Toolkit for Stateful Web Applications

News

27 April 2011 - Albatross 1.42 released

Incompatibilities

• As of version 1.42, Albatross no longer supports Python 2.3 - version 2.4 or up is required (Python 3 is not supported at this time).
• Application.handle_exception(ctx, req) no longer escapes the python and template tracebacks before executing any application-supplied trackback template. As this change could result in a potential security problem, the application-supplied exception template has been renamed from traceback.html to uncaught_exception.html. If you used this functionality, you must remove "noescape" from your traceback.html template before renaming it to uncaught_exception.html.
• SessionExpired handling was made consistent across session models and SessionExpired is now caught by the Application.run() method. When run() catches a SessionExpired exception, it calls Application.handle_session_expired(). This method attempts to display an application-supplied session_expired.html template before falling back to an internal "Your session has expired" message.
• The noescape attribute (which disables the escaping of special HTML characters) is now only honoured for tag content, not tag attributes. Formerly, noescape on a tag would turn off escaping of all attributes and tag content. On reflection, this is too broad to be used safely. The noescape attribute will now raise an exception if used on any tag other than <al-option>, <al-select>, <al-textarea> or <al-value>, and only applies to the tag content. The new htmlsafe() mechanism provides an alternative way to disable escaping of evaluated attributes (see "Escaping generated content" in the "Templates Reference" section of the manual for more information).
• The session daemon al-session-daemon and the stand-alone development server al-httpd are now part of the Albatross command line tool. Usage remains largely the same, but rather than invoking al-session-daemon start, use albatross session-server start, and rather than al-httpd module.app 8080, use albatross serve --port=8080 module.app. You can also symlink the albatross script to al-session-daemon and al-httpd. For more information on the session server, see "Session Server Daemon" in the "Guide to Building Applications" section of the manual, and for more information on the stand-alone development server, see "Stand-alone Python HTTP Server Deployment".

New Features

• Template evaluation namespaces

  Evaluation of expressions in templates normally takes place within the scope of ctx.locals. The new <al-namespace> tag allows you to switch temporarily to a subordinate namespace. Expression evaluation will take place with respect to that namespace, and input names will automatically be prefixed with the name of the namespace.

  A namespace="..." attribute has also been added to the <al-expand> and <al-for> tags allowing you to execute their contained content in the specified namespace.

• Easier selection of Application and Context classes

  A new get_app_classes(page_model, context_model) function has been added that makes choosing Application and Context classes easier. Previously, you had to choose between one of twelve Application classes and match this with an appropriate Context class. With the get_app_classes() function, you just need to choose a page model and a session model, and the appropriate Application and Context classes are returned. For more information, see the "Prepackaged Application and Execution Context Classes" section of the manual.

• Albatross command line and quick-start application skeleton

  An albatross command line utility has been added. This utility currently has four sub-commands:

  • create - creates the basic framework of an application.
  • add-page - adds a page to an application so created.
  • serve - starts a stand-alone development server for the specified application.
  • session-server - starts a session server (see "Session Server Daemon" in the "Guide to Building Applications" section of the manual for more information).

  This makes it very easy to get started writing Albatross applications:

  $ albatross create myapp
      $ cd myapp
      $ ls
      app.ini  app.py  pages	session  static
      $ ls pages
      environment.html  macros.html  new_session.py       session_expired.html
      environment.py    main.html    page_not_found.html  uncaught_exception.html
      __init__.py       main.py      raise_exception.py   unhandled_exception.html
      1$ ls static
      alxform.css  favicon.ico  logo.png  style.css
      $ albatross serve
      Starting web server for "myapp" on port 8000
      Stop the server with Ctrl-C
      http://localhost:8000/
      
  

Enhancements

• A new alternative to the noescape attribute has been implemented that allows a data source to signal that a given string does not require escaping when it is rendered by template execution by casting it to the htmlsafe() type. This moves the responsibility for this potentially risky action from the template closer to the code that generated the string. The htmlsafe() type can also be used to selectively disable the escaping of evaluated attributes.

  While the "noescape" attribute still works, it's scope has been reduced, and it should be considered deprecated.

• RandomModular applications will now attempt to render an application-supplied page_not_found.html template, before falling back to an internal "URL .. was not found on this server" message.
• A simple logging mechanism has been added. Calling the Application.set_logger(logger) method with a logger compatible with the standard python logging module registers that logger with the application. Exceptions will be logged to the logger at "error" level, and a ctx.log(msg) method allows messages to be logged at "info" level.

Bug Fixes

• If an application supplied it's own traceback.html template, the HTML headers would be emitted twice. Also see the important note above regarding escaping of the tracebacks and renaming the template.
• PageModuleMixin.load_page_module() now raises a new PageNotFound exception (a subclass of ApplicationError) if the page module is not found. This addresses a bug where the RandomPageModuleMixin would not reliably handle an unknown page.
• The standard exception handling bypasses the context logic, writing directly to the Request object. This created an issue with the session cookie not being cleared when an exception occurred. The exception handler now explicitly adds a "Set-Cookie:" header to clear the application cookie.

15 March 2010 - Albatross 1.40 released

Major Changes

• Conversion of documentation to reStructuredText - With the Python documentation moving from LaTeX markup to reStructuredText and the Sphinx documentation generator, we could no longer rely on the documentation utilities released with Python. The Albatross documentation has been converted to ReST, and uses Sphinx to render this to HTML and PDF formats.
• Albatross Forms - A simple HTML form generation and validation framework has been added to Albatross (as an optional extension under the module name albatross.ext.form).

Bug Fixes

• Redirects were not calling req.return_code() - for FastCGI deployment, this would result in the request hanging [11164].
• The python built-in __import__() gained extra arguments with python 2.6 - the decode session import hook has been changed to pass all arguments (positional and keyword) [14823].
• When rendering tags to HTML, the handling of the noescape attribute was not consistently honoured. Attribute rendering has been moved to the common Tag base class as the write_attrib() method [15398].
• Relaxed tag recognition regular expressions, so that malformed tag attributes with no value, such as <al-input name= />, are still recognised as valid Albatross tags, albeit with a null name attribute (which subsequently generates an error) [15398].

Miscellaneous Changes

• Miscellaneous changes to RedHat, Ubuntu, Solaris and OS X packaging rules.
• Reorganised the unit tests to eliminate the use of explicitly assembled test suites, relying instead on the unittest module to collect methods of TestCase subclasses. The documentation example tests were also reimplemented as TestCase subclasses, allowing them to be run via common test driver.

19 March 2007 - Albatross 1.36 released

New Features

• A "vars" attribute has been added to <al-for> - this inserts the iterator value into the local namespace. The iterator value was previously only accessible via the iterator value() method.
• <al-for> now loops over sequences using the iterator protocol by default. The previous behaviour of indexing the sequence is retained where pagination or columns are requested.
• New <al-setdefault> tag which allows macros to define default values for their arguments (suggested by Greg Bond).
• Macro expansion can now specify macro arguments via attributes on the <al-expand> tag (suggested by Greg Bond).
• Added get_param() method to Request classes, which deployment method agnostic access to http "environment" variables.

Functional Changes

• NameRecorderMixin no longer falls back to merging all request fields if no __albform__ field is present. Applications using the NameRecorderMixin and GET-style requests will need to explicitly merge relevent fields with the ctx.merge_vars(...) method.
• Pickles embedded in hidden fields are now signed using the HMAC-SHA1 algorithm (was MD5).
• The experimental FastCGI driver included in version 1.35 now becomes the default. The old driver has been renamed fcgiappold. This driver implements the FastCGI protocol itself, rather than depending on an external module.

Bug Fixes

• If the <al-option> tag was not immediately enclosed within an <al-select> tag, it would be silently ignored. <al-option> now works within containing flow control tags such as <al-for> or <al-if>, and supports attributes for dynamically setting value and label.
• Communication with the session server has been made more robust. EINTR is handled, as are partial reads and writes, and requests are restarted if the socket closes.
• Mixing radio inputs with other inputs of the same name did not raise an exception (reported by Michael Neel).
• fixed FCGI_MAX_DATA, which was incorrect due to an operator precedence mistake. Found by Tom Limoncelli.

Miscellaneous Changes

• Clean up and replace many pre-python 2.2 constructs.
• Many new tests have been added, and existing tests restructured.

20 June 2006 - Albatross 1.35 released

Note that release 1.34 was an internal release.

New Features

• A drop-in replacement for the fcgiapp module, called fcgiappnew has been added. This version implements the FastCGI protocol itself, rather than relying on an external module to implement the protocol. This new module addresses several minor problems with fcgi.py, and should be faster, although it should not be used in critical applications until it has received wider testing.

Functional Changes

• When extension tags (alx-*) are registered, their name is now checked against the template parsing regexp to ensure they can subsequently be matched.

Bug Fixes

• The AnyTag functionality was given knowledge of HTML tags for which the close tag is forbidden, so it can avoid generating XHTML empty tag (which could cause the page to fail HTML validation).
• When the disabledbool attribute was used on input tags, the disabled state was not being passed through to the input registry within the NameRecorderMixin.
• If a client closed it's connection to the session server while the server had data pending for the client, a subsequent del_write_file would generate an exception, killing the session server.

16 December 2005 - Albatross 1.33 released

• Fixed a vulnerability in the handling of submitted form fields (CVE-2006-0044, DSA-942-1).

16 August 2005 - Albatross 1.32 released

• To obtain a reference to the current frame, _caller_globals was raising and catching an exception, then extracting the tb_frame member of sys.exc_traceback. sys.exc_traceback was deprecated in python 1.5 as it is not thread-safe. It now appears to be unreliable in 2.4, so _caller_globals has been changed to use sys._getframe().
• If ctx.set_page() was called from within the start page, then the wrong page methods (page_enter, page_display, etc) would be called (those of the initial page, rather than the page requested via set_page).
• Fixes to handling of missing RandomPage page modules.

31 May 2005 - Albatross 1.30 released

• Any HTML tag can now be prefixed with "al-" allowing any attribute to be the result of python evaluation. For example:

         <al-td colspanexpr="i.span()">
        

  could produce

         <td colspan="3">
        

  and:

         <al-input name="abc.value" disabledbool="abc.isdisabled()">
        

  could produce

         <input name="abc.value" disabled>
        

• Since macros and lookups are an application global resource, they can only be defined once per application, however this was not previously checked. Redefinition of macros or lookups will now result in an ApplicationError exception.
• an in-line version of the <al-lookup> tag has been introduced, which is expanded in place if the tag has an expr= attribute.
• a new <al-require> tag has been added to allow templates to assert that specific Albatross features are available, or templating scheme version number is high enough.
• "Cache-Control: no-cache" is now set in addition to "Pragma: no-cache" - the former is defined for HTTP/1.1, the later for HTTP/1.0.
• Simplified session cookie handling.
• FastCGI apps were not being explicitly finalised, relying instead on their object destructor, with the result that writing application output (or errors) would be indefinitely deferred if object cycles existed. We now call "fcgi.Finish()" from the fcgiapp Request.return_code() method.
• When handling exceptions, the traceback is now explicitly deleted from the local namespace to prevent cycles (otherwise the garbage collection of other objects in the local namespace will be delayed).
• Two fixes to the <al-select> tag: the albatross-specific "list" attribute was leaking into resulting markup, and the use of the "expr" attribute would result in invalid markup being emitted.
• Thanks to Robert Fendt for picking this up: the Albatross-generated hidden field input element must not appear naked inside a form element for strict HTML conformance. The solution is to wrap the input elements in div.
• BranchingSession sessions could not be "deleted" - the solution is to add a dummy "session" shared by all branches, which is deleted when one branch "logs out".

19 October 2004 - Albatross 1.20 released

Changes in this release include:

• A new BranchingSessionContext application context class, that provides better syncronisation between browser state and application state when the application state is stored server-side.
• Improved adapters for the different deployment modes (CGI, FastCGI, mod_python, standalone, etc). Application status codes (such as HTTP_INTERNAL_SERVER_ERROR) are now returned to the browser.
• The page module loader in PageModuleMixin has been reimplemented so that it does not pollute sys.modules. Page modules are now loaded into a synthetic module's namespace, rather than the global module namespace. This will break code that defined classes in page modules and placed instances of those classes into the session.
• Multi-instance response headers are now supported (so, for example, more than one Cookie can be set. The ResponseMixin.get_header() method now returns a list rather than a single string.
• ctx.req_equals(name) now checks for name.x if name is not found. This makes using image maps as buttons easier.
• Under some circumstances, redirect_url() would redirect to incorrect or invalid URLs (for example, an https app would redirect to http) - the URI parsing has been refactored, and this bug has been fixed. Tests were also added for the refactored URI parsing.
• Several cookie handling bugs were fixed.

20 July 2003 - Albatross 1.10 released

This release contains many new features, most of which are described below.

• Matt Goodall developed support for deployment of applications via FastCGI. FastCGI applications import their Request class from albatross.fcgiapp.
• Matt Goodall developed support for standalone deployment of applications via the standard Python BaseHTTPServer module. The al-httpd program can be used to deploy a CGI application as a standalone BaseHTTPServer server.
• All Albatross exceptions have been redefined to indicate the source of the error; user (UserError), programmer (ApplicationError), or Albatross itself (InternalError). The ServerError exception reports problems related to the session server, SecurityError reports either a programmer error, or a user attempt to access restricted values in the execution context, and TemplateLoadError reports failures to load templates. All of the exceptions inherit from AlbatrossError.
• All response header management has been moved to the ResponseMixin class. The AppContext class now inherits from ResponseMixin. The Request class no longer tracks whether or not headers have been sent.
• The Application.run() method no longer unconditionally returns an HTTP response code of 200. The returned response code is retrieved from the Request.status() method. You can call the Request.set_status() method to override the default HTTP response code of 200.
• The <al-input> tag now supports type="file" input tags. When you use file input tags the enclosing <al-form> tag automatically adds the enctype="multipart/form-data" attribute to the generated <form> tag.
• The Application.run() method now saves the session before flushing the response to the browser. This allows applications to support dynamically generated images.
• The exception formatting in Application.handle_exception() has been moved into the format_exception() method. Applications can perform their own exception formatting and/or send formatted exceptions to locations other than the browser.
• The ExecuteMixin.write_content() method now converts unicode to UTF-8.
• During NamespaceMixin.eval_expr() the execution context is temporary placed into the local namespace as the variable __ctx__.
• The <al-lookup> tag now registers the lookup during template parsing rather than during evaluation. This allows template code to make use of lookups that are defined later in the same file. The item dictionary is created the first time that the lookup is used rather than when the template is interpreted.
• Macro arguments are now evaluated when they are referenced rather than before they are passed to a macro. This removes a limitation where you could not define macros including <al-form> tags that retrieved <al-input> tags from their arguments. Previously the <al-input> tags passed as macro arguments would have been evaluated outside of the context of the form defined in the macro. This effectively made the input tags invisible to the form recorder.
• The templates reference documentation has been completely restructured to improve clarity. All attributes of each tag have been documented.

28 October 2002 - Albatross 1.01 released

Changes in this release include:

• Prevent variables starting with an underscore from being merged into the context local namespace from request objects. This prevents attackers spoofing variables such as __page__, and any user variables such as authentication data.
• Updated documentation to explain underscore prefixed names in execution context are protected from browser modification.
• Applied bugfix from Detlef Lannert - null attributes would raise a TypeError (for example: <AL-IMG EXPR="..." ALT="">). Once fixed, this exposed another bug where an attribute with a null value would lose it's value altogether (XHTML requires attributes to have a value, null or otherwise).
• Exception text is now escaped before being sent to browser.
• The PageModuleMixin does not reload a page module if was previously loaded by unpickling the session. This is to avoid a problem with new code in the Python 2.2 pickle/cPickle module which causes the following to fail.

  import pickle, copy
  o = copy._EmptyClass()
  reload(copy)
  pickle.dumps(o, 1)
     

• Pickling errors are now trapped by looking for the PickleError exception.
• Bug prevented nameexpr from being used in <AL-TEXTAREA> and <AL-SELECT>.
• Radio input was converting "value_attr" to str(), but not "value", which was causing the "checked" test to fail if value was anything but a string. Also changed checkbox input to convert both to strings before doing the comparison.

20 August 2002 - Albatross 1.00 released

Changes in this release include:

• PageModuleError is raised when a page module is loaded which does not define one of page_enter(), page_leave(), page_process() or page_display().
• Application merge_request() method now defers browser request merging to the execution context. This allows the RecorderMixin class to define the correct request merging process.
• Include filename in template load errors - when templates are loaded by any means other than <al-include>, the filename was often not in the traceback.
• When you use FormRecorderMixin you must now specify LIST attribute on <AL-INPUT> and <AL-TEXTAREA> tags if more than one field in a form has the same name. This also causes the request merging to always place those input values into a list in the local namespace regardless of how many values were sent by the browser. FieldTypeError is raised when LIST attribute value is inconsistent with multiple/singular instances of an input field in a form.
• Replaced all relative module imports with absolute imports to avoid namespace clashes.
• Several SessionFileAppMixin portability fixes - use "rb" and "wb" mode on open of session files for windows portability, and improvements to handling of platforms without /dev/urandom.
• albatross.random module renamed to albatross.randompage.
• SessionFileAppMixin now checks that the session-id returned from the browser does not contain any filesystem path components that could be used to access files outside the session file directory.
• Tag constructors are now passed the execution context as the first argument, which makes them consistent with other methods, and allows greater flexibility in code placement.
• Macros are now registered at load time, rather than execution (to_html) - this allows forward references, and other tricks.
• New "ellipsis" mode for tree iterator, which are a variant of lazy trees where nodes at shallower levels are progressively collapsed into ellipses as the user opens deeper nodes. The user can reopen the collapsed nodes by selecting an ellipsis.
• TreeIterators now support get_open_aliases(), get_selected_aliases(), set_open_aliases(), set_selected_aliases() methods to set and get open and selected nodes for the TreeIterator. This allows you to enter a page with tree nodes already open or selected, and also to save the state of the tree for future use.
• Added a basic /etc/init.d style rc script for the TCP session server. The session server script is now installed to you scripts directory (typically /usr/local/bin). Session server component modules simpleserver and pidfile are now installed into the albatross module directory, along with the rest of albatross.

15 March 2002 - Albatross 0.06 released

Changes in this release include:

• The template parse has been enhanced to handle attribute values enclosed with either single or double quotes. The enclosing quote character can be used in the attribute string if it is escaped by a backslash ('\'). Attribute values can be broken over multiple lines.
• All pickles sent to browser are now MD5 signed to detect and prevent modification. New pickle security techniques can be defined by defining a replacement for the PickleSignMixin class.
• The new RandomPageModuleMixin class now allows you to build applications which use the request URL to determine which page module will be used to process the browser request.
• A set of classes for implementing file based server side sessions has been added.
• The simple session server can now be run as a daemon under Unix.
• Program mainline for mod_python deployment has been simplified. You now hook into mod_python like this:

  def handler(req): 
      return app.run(Request(req))

  instead of this:

  from mod_python import apache 
      :
  def handler(req): 
      app.run(Request(req)) 
      return apache.OK

• New redirect() execution context method for doing browser redirection.
• Session saving can be disabled via the new set_save_session() method.
• The samples have been completely reorganised and documentation updated to make them less confusing. An installation program has been provided to reduce the pain of running the samples.
• Many documentation improvements.

14 January 2002 - Albatross 0.05 released

Changes in this release include:

• Many documentation fixes.
• Documentation for macro tags has been completed.
• New calendar example extension tag.
• Fixed infinite recursion bug in nested macro expansion.
• All interactive examples in documentation are now validated via 'make test' in the doc directory.
• Lots of new unit tests.

26 December 2001 - Albatross 0.04 released

First public release of Albatross.