XML

Google Maps and XSL

Simon Willison — 2005-02-08T14:19:37-00:00

I'll probably write more on this later, but it seems that Google Maps is using XSL. I spotted it loading the following pages while sniffing its activity with LiveHTTPHeaders:

This is in addition to the (now expected) XMLHttpRequest stuff. There even appears to be some of Microsoft's weird VML, although as I'm on a Mac I don't have access to IE/Windows to see what it's doing with it.

The bulk of the Google Maps JavaScript appears to be hidden away in maps.1.js, which becomes a lot more readable if you feed it through PrettyPrinter.de

As for Google Maps itself, it's an amazing piece of work but it's a shame they didn't follow map.search.ch's lead in degrading gracefully to a static version for unsupported browsers.

If anyone has any further insights in to how it all works, please post them in a comment.

Staying valid

Simon Willison — 2004-05-02T21:55:53-00:00

Andrei Herasimchuk:

There seems no automatic way to keep a site valid with web standards unless you close it off to the rest of the world to contribute to it. I will not do that anytime soon.

There is: I'm doing it. Next Thursday will mark the one year anniversary of my switching to application/xhtml+xml as the content-type header for this site, for user agents that support it. Using that content-type forces Gecko engine browsers to refuse to render pages if they are not well-formed XML, so if a page is invalid I hear about it pretty quickly.

So how do you keep a frequently updated site with data from external sources and user comments valid? There are really only two things you need to do. Firstly, ensure that everything going IN to the system (entries and comments) is valid XHTML. I do that using a simple validation system for comments and a bookmarklet for my own entries. Secondly, any and all data from external sources (my blogroll from blo.gs, blogmark URLs added using a bookmarklet) needs to be entity-escaped before being displayed on the site. In my case, a call to PHP's htmlspecialchars() function is all that's needed.

I'm not saying my system is ideal - the need for well formed markup in comments is a major usability issue even on a site with an audience consisting mostly of web developers. But it's certainly possible to operate a site in XHTML with frequent updates and user comments while staying valid at the same time.

A myriad of markup systems

Simon Willison — 2004-04-13T04:58:54-00:00

It's hard to avoid the legions of custom markup systems out there these days. Every Wiki has it's own syntactical quirks, while packages like Markdown, Textile, BBCode (in dozens of variants), reStructuredText offer easy ways of hooking markup conversion in to existing applications. When it comes to being totally over-implemented and infuratingly inconsistent, markup systems are rapidly catching up with template packages. Never one to miss out on an opportunity to reinvent the wheel, I've worked on several of each ;)

My most recent markup handling attempt has just been published as part of my SitePoint article on Bookmarklets (cliché). It's a structured markup language in a bookmarklet: activate the bookmarklet to convert the text in any textarea on a page to XHTML. The syntax is ridiculously simple, and serves my limited needs just fine:


= This is a header

Here is a paragraph.

* This is a list of items
* Another item in the list

Converts to:


<h4>This is a header</h4>

<p>Here is a paragraph.</p>

<ul>
 <li>This is a list of items</li>
 <li>Another item in the list</li>
</ul>

The algorithm is simple, and easily portable to any language you care to mention:

Normalise newlines to \n, for cross-platform consistency.
Split the text up on double newlines, to create a list of blocks.
For each block:
1. If it starts with an equals sign, wrap it in header tags.
2. If it starts with an asterisk, split it in to lines, make each a list item (stripping off the asterisk at the start of the line if required) and glue them all together inside a <ul>.
3. Otherwise, wrap it in a  tag provided it doesn't have one already.
Glue everything back together again with a couple of newlines, to make the underlying XHTML look pretty.

The bookmarklet comes in two flavours: Expand HTML Shorthand (the full version) and Expand HTML Shorthand IE, which loses header support in order to fit within IE's rippling 508 character limit. A more capable bookmarklet could be built using the import-script-stub method described in my article, but the implementation of such a thing is left as an exercise for the reader (I've always wanted to say that).

Incidentally, there's a very common bug in markup systems that allow inline styles that proves extremely difficult to fix: that of improperly nested tags. Say you have a system where *text* is bold and _text_ is italic; what happens when the user enters _italic*italic-bold_bold*? Most systems (and that includes Markdown, Textile and my home-rolled Python solution) use naive regular expressions for inline markup processing and will output vadly formed XHTML: italicitalic-boldbold. To truly solve this problem requires a context-sensitive parser, which involves an unpleasantly large amount of effort to solve what looks like a simple bug.

XML highlights for PHP 5

Simon Willison — 2003-12-20T23:44:53-00:00

XML in PHP5: An in-depth look into advanced XML features (via Keith) does exactly what it says on the tin. Here are the bits that caught my eye:

HTML Support in ext/xml - PHP 5 can load in not well-formed HTML documents and create a DOM tree from them.
XPath support - and it works with HTML loaded via the above.
XML Validation, including support for RelaxNG! I wonder if they'll support compact syntax.
Extending DOM Classes - this is really cool, and demonstrates how much more mature PHP 5's OOP support is.

Unforuntately, my biggest criticism of PHP remains: all of the above is supported using functions built in to the default namespace! The lack of a smart namespace system (like Python's modules) really gets in the way when you start trying to write reusable code or large applications.

RELAX NG now an ISO standard

Simon Willison — 2003-12-16T03:41:03-00:00

Via James Robertson, RELAX NG (altogether too many caps) has been published by ISO as an International Standard. RELAX NG compact syntax is will hopefully join it soon.

RELAX NG is the sane alternative to XML Schema. It's a schema language which allows you to define the "shape" of a class of XML documents - a bit like a DTD but far more flexible. The reason I like it is the compact syntax, which lets you define schemas using a non-XML syntax similar in some ways to CSS. This makes defining new types of XML document far less unpleasant.

XUL in Safari

Simon Willison — 2003-10-26T01:20:57-00:00

Safari 1.1 is included with the new release of Mac OS X, Panther. From Dave Hyatt's list of Safari 1.1 features:

A complete implementation of the XUL box model. Safari on Panther supports the complete XUL box model, including horizontal and vertical boxes, the ability to flex, and the ability to reorder content and reverse content. If you're building canned content that you control using WebKit, you'll find a whole new range of layout possibilities at your disposal. Need to create dynamically sized headers and footers and flexible center content? The XUL box model can do that. Need to center an object within the viewport? The XUL box model can do that too.

With Microsoft's alternative to XUL seemingly a few years away, are Apple looking to beat them to it with an implementation that's compatible with Mozilla?

Microsoft's XUL

Simon Willison — 2003-10-24T15:59:41-00:00

According to this Microsoft blogger the reason IE development has seemed quiet of late is that they've been working hard on XAML, an XML Application Markup Language which is a way to create applications in the browser (or out for that matter) [...] It is basically an XML structure with CSS and JavaScript. The CSS defines the appearance and the JavaScript dictates behavior.

Sound familiar? It should do - that's exactly what XUL does. On the plus side, this completely validates the grossly under-appreciated importance of Mozilla's biggest secret. On the negative side, this looks set to represent the ultimate browser lock-in - in a few years time when IE 7 comes as standard on new PCs I wouldn't be surprised to see the corporate software development world moving almost exclusively to this technology - after all, it's going to be extremely easy both to develop and to distribute and it will have all of the benefits of a web application without the downside of the restricted GUIs offered by HTML.

Of course, Microsoft might make it an open standard. But then I might win the lottery next week.

I wonder if it will be possible to transform XAML in to XUL using XSLT?

Update: It has been pointed out that the author of the above blog entry is not a Microsoft employee.

Using XPath to mine XHTML

Simon Willison — 2003-10-21T05:31:23-00:00

This morning, I finally decided to install libxml2 and see what all the fuss was about, in particular with respect to XPath. What followed is best described as an enlightening experience.

XPath is a beautifully elegant way of adressing "nodes" within an XML document. XPath expressions look a little like file paths, for example:

/first/second: Match any <second> elements that occur inside a <first> element that is the root element of the document
//second: Match all <second> elements irrespective of their place in the document
//second[@hi]: Match all <second> elements with a 'hi' attribute
//second[@hi="there"]: Match all <second> elements with a 'hi' attribute that equals "there"

A full XPath tutorial is available.

The Python libxml2 bindings make running XPath expressions incredibly simple. Here's some code that extracts the titles of all of the entries on my Kansas blog from the site's RSS feed:

>>> import libxml2
>>> import urllib
>>> rss = libxml2.parseDoc(
      urllib.urlopen('http://www.a-year-in-kansas.com/syndicate/').read())
>>> rss.xpathEval('//item/title')
[<xmlNode (title) object at 0xb4b260>, <xmlNode (title) object at 0xa99968>, 
<xmlNode (title) object at 0x10dce68>]
>>> [node.content for node in rss.xpathEval('//item/title')]
['Music and Brunch', 'House hunting', 'Arrival']
>>>

Why is this so exciting? I've been saying for over a year that XHTML is an ideal format for storing pieces of content in a database or content management system. Serving content to browsers as HTML 4 makes perfect sense, but storing your actual content as XML gives you the ability to process that content in the future using XML tools.

So far, the best example of a powerful tool for manipulating this stored XML has been XSLT. XSLT has its fans, but is also often criticised as being unintuitive and having a steep learning curve. XPath is a far better example of a powerful, easy to use tool that can be brought to bare on XHTML content.

Enough talk, here's an example of what I mean. The following code snippet creates a Python dictionary of all of the acronyms currently visible on the front page of my blog, mapping their shortened version to the expanded text (extracted from the title attribute):


>>> blog = libxml2.parseDoc(
    urllib.urlopen('http://simon.incutio.com/').read())
>>> ctxt = blog.xpathNewContext()
>>> ctxt.xpathRegisterNs('xhtml', 'http://www.w3.org/1999/xhtml')
0
>>> acronyms = dict([(a.content, a.prop('title')) 
    for a in ctxt.xpathEval('//xhtml:acronym')])
>>> for acronym, fulltext in acronyms.items():
	print acronym, ':', fulltext


DHTML : Dynamic HyperText Markup Language
URL : Universal Republic of Love
HTML : HyperText Markup Language
SIG : Special Interest Group
PHP : PHP: Hypertext Preprocessor
CSS : Cascading Style Sheets
>>>

The above code is slightly more complicated than the first example, as using XPath with a document that uses XML namespaces requires some extra work to register the namespace with the XPath parser. Still, it's a pretty short piece of code considering what it does.

For an example of how powerful XPath can be on a much larger scale, take a look at Sam Ruby's XPath enabled blog search feature.

Unstructured linkage

Simon Willison — 2003-10-07T16:09:57-00:00

Tom Gilder: Absolutely fucking nuts
Get ready for IE changes
Score Higher in Google Search Engine (and why Google is saving the web)
High performance XML-RPC
Juggling with packets: floating data storage (via Ned)
Taking the Pulse of XML Editing
Beware the dangers of generic Exceptions
Blogging for Dollars
Resign Patterns: Ailments of Unsuitable Project-Disoriented Software
PatchDayReview.com - Explaining Microsoft's Weekly Patches in Non-Technical Terms
XML Application Objects (via the SitePoint Forums)
Chapter 17 of the XML Bible: XSL Transformations
Radiohead, as drawn by 5th graders
Telephone Preference Service, the UK Do Not Call list

New content management blog

Simon Willison — 2003-09-15T11:06:25-00:00

Ideas in Technology and Publishing is a great new blog covering content management, XML and other publishing related technologies. It's less than a month old so it's still possible to read through the archives in full, which I've just done and recommend to anyone with an interest in content management.

XML textarea validation bookmarklet

Simon Willison — 2003-08-27T07:27:31-00:00

Jesse Ruderman's Blogidate bookmarklets cycle through all of the textareas on the current page and submit their contents for validation. I suggested an alternative approach, and to my great delight has has followed it up with the essential Blogidate XML well-formedness. One click, and each textarea on the page will be checked to see if it contains valid XHTML. If it does, the background goes green - if not, it goes red. Hopefully I'll never post another invalid entry (my previous solution broke when I switched to application/xml+xhtml).

Note to self

Simon Willison — 2003-08-13T17:20:27-00:00

When writing an XML parsing class in PHP, don't forget the ampersands in the following code snippet:

xml_set_element_handler($parser, array(&$this, '_tagOpen'), array(&$this, '_tagClose'));
xml_set_character_data_handler($parser, array(&$this, '_cdata'));

Failure to include them can lead to two days of debugging, frustration and hair loss. Solution found thanks to Keith's PHP rant (which itself reads like the result of several days of hell). Roll on PHP 5, where objects are passed by reference by default.

Moveably Type with XSLT

Simon Willison — 2003-08-11T19:13:35-00:00

Kevin Davis has set up an impressive demonstration of the power of Moveable Type templates when combined with browser-side XSLT transformations. He's set up MT to output an XML document containing just his entries (similar to an RSS feed), along with a link to an XSLT stylesheet that causes Mozilla and IE 6 to transform the entry and render it as XHTML.

PHP XPath Implementation

Simon Willison — 2003-07-28T23:59:35-00:00

This looks like it could be really useful: an XPath implementation in pure PHP (no extra modules required), via More Like This.

XML