Browsers

Graphing requests with Tamper Data

Simon Willison — 2006-10-17T18:21:06-00:00

I spent the weekend in Boston, speaking at GBC/ACM's Deep Ajax seminar with Alex Russell and Adrian Holovaty. I'll be posting some notes on this later, but I wanted to share a really neat Firefox extension that Alex showed me: Tamper Data.

Tamper Data is an extension for intercepting HTTP requests and modifying them. I have very little interest in this functionality myself, but hidden deep within the extension is the ability to do this:

That's a graph showing what happens when you load up www.yahoo.com. It shows every component of the page - JavaScript, CSS, images - and when each component started and finished loading. You can use it to get an idea for how long it took between the HTML starting to load and the browser beginning to pull in the CSS, then the images, and so on. It's a superb visualization of what happens when a page is loaded.

Unfortunately, if you install and run the extension (Tools -> Tamper Data) you'll see this instead:

To get the graph, you have to right click in the main data grid and select "Graph All" from the context menu. Be sure to hit "clear" before loading a page that you want to graph or you'll end up seeing data from other pages too (you should shut down GMail or similar to prevent their polling requests from polluting the graph).

It's a great tool but it's pretty well hidden. If you're looking for a side project, implementing the same functionality in a smaller extension (maybe as an extra tab in the Page Info screen) would be a significant service to the web development community.

Sticking with Opera 9

Simon Willison — 2006-08-06T18:32:57-00:00

It's been a month and a half since I started using Opera 9, with a promise to report back later. I'm still using it, although some of the things I liked initially have faded while others have emerged.

Firstly, the ability to browse cached documents offline turned out to be a dud. The behaviour I was observing (where I could view pages from my cache while disconnected) was due to the following setting, in Preferences -> Advanced -> History:

As someone whose work involves making websites the idea of seeing stale documents for five hours is pretty horrifying, so I changed that to "check documents never" as soon as I saw it (I'm pretty confident that doesn't affect conditional-GET, which should work regardless). As soon as I did that I lost the ability to browse offline. Browser vendors take note: you can still be the first modern browser to implement a proper offline mode!

The disadvantage of changing those setting is that they can break JavaScript image preloaders, leading to an unsettling flicker when you mouse over some links. That's why I've left my image setting above to check every five minutes.

Despite losing offline browsing, other features have popped up that have kept me happy:

Opera is the only Mac browser I've used that includes a working full-screen mode; great for presentations.
Hitting space moves you down the height of the viewport, as with other browsers. Hitting space at the bottom of the page takes you to the next page in the 'sequence', if there is one. My first guess was that this used <link rel="next">, but it also works on Yahoo!, Technorati and Google search results pages which don't have those links. I suppose it's looking for any link on the page with 'Next' as the link text. Whatever it's doing, it works surprisingly well.
Site compatibility really is excellent; I run in to sites that don't work in Opera about as often as I do sites that don't work in Firefox. Part of that might be due to Opera's ingenious browser.js file, which includes Greasemonkey-style site-specific fixes and automatically updates itself once a week. Clever, but a bit scary at the same time.
The built-in IRC client is good as well - if it wasn't for Colloquy it would be my first choice for IRCing on the Mac.
Finally, Opera's low memory footprint continues to keep me from switching back to Safari or Firefox. My Mac (a two and a half year old PowerBook) is noticeably less sluggish now that those two applications have been relegated to JavaScript hacking (gotta love FireBug) and the occasional misbehaving site.

I haven't even touched Opera Mail yet, and the gadget support is something I played with once and never used again (but then I don't use Dashboard that much either).

Overall I'm extremely happy with Opera 9 and I'd recommend giving it a go - especially if you've tried and disliked an older version.

Two revolutionary features in Opera 9

Simon Willison — 2006-06-20T23:42:18-00:00

Wow, if I'm not careful this is going to turn in to a promotional blog for Opera.

So, I've been playing with Opera 9 which came out earlier today (in a dual release for Windows and Mac, which is refreshing). It's an impressive package - it's fast, it renders every page I've throw at it so far, it passes Acid 2, supports Canvas and SVG and has a neat widgets implementation. Performance on Google Maps is a bit sluggish but other than that it's been extremely snappy. The tab screenshot thumbnails are a nice touch.

There are two features however that set it head and shoulders above its competitors in terms of raw utility:

You can browse cached documents offline! I'm not sure when other browsers forgot how to do this but it's been years since I've been able to unplug from the network and still view pages that have been saved in my cache.

In fact, this feature is almost too effective: I had to keep double-checking to make sure my WiFi was definitely turned off. A visual indicator that you are viewing an offline document would be a very useful addition.
You can quit your browser and reopen it to the same state. Other browsers have been slow to cotton on to the fact that my browser state is important data - I frequently leave interesting pages open for days at a time and a browser crash causes me real pain. Safari and Firefox both have extensions that enable this but it's great to see it built in to the core product.

Sadly, Opera doesn't persist partially completed form data (so you'll still lose that half-written blog entry if you weren't smart enough to write it in a text editor and paste it in at the last moment) but it's still a huge improvement over the rest.

As far as I can tell those two features have been in Opera since before the current release, but I'd never noticed them before. Here's hoping other browser manufacturers follow suit.

While the interface is leaps and bounds ahead of Opera 6/7, it still has a few crufty edges: the preferences are hard to navigate, the text in the tab screenshot window comes across as an afterthought, there's a randomly placed recycle bin in the upper right hand corner and it took me a while to find a skin I liked (tango cl). The default behaviour for command-clicking a link is to open a new tab and switch to it; my preference is for opening in the background but I can't work out how to change that option (though shift-command-click does what I want). Overall though it rates extremely well.

As usual, it's worth looking over the detailed changelogs (windows, mac). There are some real gems for scripters; Here's the list in full:

Added support for XSLT 1.0 and the XSLTProcessor constructor.

Added support for XPath 1.0.

Added support for DOM level 2 Style Sheets and associated parts of DOM level 2 CSS.

Implemented designMode for rich text editing.

Added support for the CONTENTEDITABLE attribute and contentEditable property.

Implemented support for canvas, as described in the Web Applications 1.0 draft, as well as the opera-2dgame context.

Added support for Audio, as described in the Web Applications 1.0 draft.

Multiple improvements to XMLHttpRequest support.

Added support for onmousewheel events.

Added support for document.load and document.adoptNode.

Added window.getSelection and associated methods.

Improved handling of offsetTop, offsetLeft, and offsetParent.

Removed support for "javascript:" URLs in CSS.

Pages using certain JavaScript events will reload when visited in history. A knowledge base article is available.

I'm going to use Opera 9 as my default browser for the next week or so to see how well it holds up. I'll report back on the experience later.

Opera Mini 2.0

Simon Willison — 2006-05-11T22:12:25-00:00

Just as I was getting thoroughly sick of the whole X-2.0 trend along comes a product I can really get excited about. Opera Mini 2.0 is a truly lovely piece of software. It's a free web browser for your phone, accompanied by a free proxy:

When surfing with Opera Mini, Web pages are optimized and compressed before being sent to your phone. This means that even though your mobile provider may charge you for the data which is transferred to your phone, the amount of data transferred is significantly less than it would normally be, making mobile surfing cheaper.

Most UK mobile data plans are pretty extortionate. I was quoted over £500 a month for an unlimited plan recently; compare that to Leonard Lin who pays $15 a month in the US. Having a compressing proxy is essential.

In stark contrast to the desktop edition the user interface is beautifully simple, relying mostly on the joypad to navigate with a full-screen editor for entering URLs and filling in forms. Opera's small screen rendering technology is used to linearise the page content while keeping background colours and images intact.

The one feature I'd love to see added is a "reload page with images" menu option. I generally browse with images turned off, and viewing a page with its images requires me to navigate to the settings screen, toggle images on, reload the page and then remember to turn images back off again afterwards.

Opera Mini doesn't appear to support JavaScript, but despite that nearly all the sites I've visited have been perfectly usable. Even Gmail works, thanks to an automatic fallback to their plain-old-HTML interface. With any luck it will become part of the accepted accessibility benchmark - I know I'll be testing sites with it in the future.

A quote

Simon Willison — 2004-12-16T15:57:38-00:00

Ramez Naam, group program manager for MSN Search, declined to say whether or not search functions would be integrated directly into Microsoft's Internet Explorer. But a Microsoft executive, who asked to remain unnamed, told me that his company had recently reconstituted its browser development organization. "Microsoft effectively disbanded the Internet Explorer group after killing Netscape," he said. "But recently, they realized that Firefox was starting to gain share and that browser enhancements would be useful in the search market."

From What's next for Google, by Charles H. Ferguson.

Time to fix those broken pages

Simon Willison — 2004-05-29T23:46:45-00:00

I have a whole bunch of gripes about Internet Explorer, but my personal favourite is the way it will render a document served with a text/plain Content-Type header if it thinks the file might contain HTML. The direct result of this is that people with misconfigured web servers who are serving their HTML with the wrong Content-Type frequently don't realise, so when users of better behaved browsers such as FireFox visit they get hit in the face with a page of raw source code.

The times they are a-changing. I just spotted this gem in MSDN's article How to Make Your Web Site Work with Windows XP Service Pack 2:

Q: Does your Web site contain files with file types that do not match their Content-Type and/or file extension?

A: You should correct all of these mismatches. Both the Content-Type and the file extension must match the type of the file for a download prompt to appear. Be sure this is true for your Web pages as well. If the Content-Type is plain/text, then they will not render as HTML.

Of course, the rate at which people upgrade to service pack 2 is likely to be pretty poor but at least new machines will have it installed by default. Hopefully sites serving the wrong Content-Type for their HTML documents will be forced to clean up pretty quickly.

The other issue mentioned in that quotation - forcing the file extension to match the Content-Type - is a little odd from a non-Windows OS point of view but I'm sure there's a rational reason behind it. At the end of the day, anything that improves Windows security is a good thing for the health of both the Internet and society in general.

Automatic line ending conversions in IE

Simon Willison — 2004-02-17T02:03:14-00:00

I've just updated my SitePoint blog with a tale of Javascript debugging woe. To cut a long story short, Internet Explorer for both Mac and Windows automatically converts sane line endings in to the platform specific alternatives whenever you assign a Javascript string to the value attribute of a text area. It's the kind of quirk that can take up a whole morning's worth of debugging.

No more usernames in URLs

Simon Willison — 2004-01-30T08:14:25-00:00

This one could get very interesting. Microsoft have announced that an upcoming update to Internet Explorer will remove the ability to include usernames in URLs completely. This is in response to the growing problem of so called "phishing" scams, which use trick URLs to con important information such as passwords and credit card details out of unsuspecting browser users.

Phishing is big business. In this article on SecurityFocus, a loose transcript is provided of a talk by an FBI agent who explains how phishing is used by organised crime gangs in Eastern Europe:

This is bad enough and it's also cruelly funny, but the scary part came in when Dave started talking about the other group behind the explosion of viruses and Trojans: Eastern European hackers, backed by organized crime, such as the Russian mafia. In other words, the professionals.

These people are after one thing: money. The easiest way to illegally acquire money now is through the use of online tools like Trojans, or through phishing: set up a fake Web site for PayPal or eBay or Amazon, and then convince the naive to enter their usernames, passwords, and credit card information. Viruses and spam also intersect in this nasty spiderweb. Viruses help spread Trojans, and Trojans are used to turn unsuspecting users' computers into spam factories, or hosts for phishing expeditions, and thus furthering the spread of all the elements in this process: viruses, Trojans, spam, and phishing. It's a vicious cycle, and unfortunately, it appears to be getting worse. The FBI is working as hard as it can, but the nations of Eastern Europe are somewhat powerless to solve the problem at this time.

IE is so susceptible to this kind of attack that it's not even funny. In addition to the "invisible username" bug I covered last month, a recent discovery compounds the problem by allowing dangerous executable files to pose as safe file types when downloaded from the web. New Explorer hole could be devastating has the full details.

Microsoft's solution is drastic to say the least. Passing the username as part of a URL has been part of the makeup of the internet since at least 1994, and the ability is baked in to a huge range of web client and server software. It's described in RFC 23996. The feature is rarely used however, and the overall effect of its removal from IE is hard to judge. Off the top of my head I can think of only one site that uses it for legitimate reasons: FilePlanet, which incorporates it in to the site's download queuing system (at least last time I checked).

There's an interesting contrast to be made here between open and closed development methodologies. The Mozilla project has had a bug open on this issue for over two years, which has drawn over 170 comments with plenty of great ideas but no approved solution. Microsoft on the other hand have remained silent on the issue until (we can only assume) the bad publicity surrounding it forced them to act, at which point they announced a fix that appears to gly in the face of commonly accepted web standards - but does undoubtedly solve the problem. Of course, with no chance for user feedback prior to the decision it amounts to little less than a decree from God - which correlates directly to their inarguable domination of the browser market, at least in terms of market share.

Of course, the millions of IE users who decline to upgrade their browser will remain just as susceptible as they always were (unless they stop clicking links) - a fact for which we can hardly blame Microsoft. It does however mean that phishing will remain a lucrative scam for a long time to come.

You mean there IS an IE team?

Simon Willison — 2004-01-15T04:50:36-00:00

Robert Scoble went to lunch with the head of the Internet Explorer team - yes, they still exist, despite having released nothing but security patches for over two years.

Robert says that the team is looking to work with community members to improve Internet Explorer. If this means dialogue, or even some feedback on what they're up to, this is a very good thing. Less promising is Robert's follow-up comment on the topic of improved CSS and PNG support (by far the most requested developer feature):

Another thing that the commenters generally aren't thinking of is "how to get adoption." I keep pointing out that if we fixed the CSS and PNG issues, you still wouldn't be able to use those for years. Why? Cause consumers (and companies) really don't care about those issues and won't download a new version just cause you fixed one or two issues.

Believe us, we know. That's why we want these issues to be fixed as soon as possible - so we can deploy sites that take advantages of these standards before the oil runs out and human civilisation disintegrates in to a mass of warring tribes more interested in canibalism than visually appealing web sites.

Javascript debugging: IE Option gotcha

Simon Willison — 2003-12-13T21:26:12-00:00

I've been debugging Javascript today. I like Javascript as a language, but I doubt anyone would disagree that it's a horrible, horrible language to debug across multiple browsers. Firebird at least has good debugging support - I currently use the Javascript Console and Jesse Ruderman's shell bookmarklet and I really need to learn to use Venkman some day. If anyone knows a better way of debugging Javascript in IE than relying on the lame popup box I'd love to hear about it.

In any case, one bug that I successfully vanquished today involved the humble <option> element. My HTML looked something like this:

<select name="categories" multiple="multiple" id="catfrom" size="20">
 <option>entertainment</option>
 <option>movies</option>
 <option>sports</option>
 <option>news</option>
</select>

I was having all kinds of strange problems in IE, which I finally tracked down to the following gotcha: in IE, the value property of an Option object is empty if the corresponding option tag doesn't have a value attribute. This is counter-intuitive because in HTML if an option attribute is omitted the text inside the option element is used as the value instead. Mozilla browsers duplicate this in their handling of the Option object; IE doesn't. Hopefully this tip will save someone some debugging time in the future.

Update: David Lindquist pointed me to Microsoft's free Script Debugger.

Nasty new IE vulnerability

Simon Willison — 2003-12-09T19:21:45-00:00

Most people reading are probably aware of the common trick whereby spammers and other assorted ne'er-do-wells publish URLs with usernames that look like hostnames to fool people in to trusting a malicious site - for example, http://www.microsoft.com&session%123123123@simon.incutio.com. This trick is frequently used by spammers to steal people's PayPal accounts, by tricking them in to "resetting" their password at a site owned by the spammer but disguised as PayPal.com.

Today's new Internet Explorer vulnerability makes the problem a hundred times worse. By including an 0x01 character after the @ symbol in the fake URL, IE can be tricked in to not displaying the rest of the URL at all. Don't expect a patch for a while either; the guy who discovered the bug released it to BugTraq on the same day he notified the vendor.

Browser testing utopia

Simon Willison — 2003-11-11T01:05:23-00:00

Made possible by Ryan Parman. Now if I had a Mac and Virtual PC, I'd be able to run even more...

Multiple Internet Explorers

Simon Willison — 2003-11-07T00:46:56-00:00

In possibly the best news web designers will hear this year, Joe Maddalone of Insert Title Web Designs has discovered a way of running multiple versions of Internet Explorer on one installation of Windows! The problem of testing in different versions of IE has plagued developers for years, and it's fantastic to see a solution that doesn't involve running multiple partitions with separate Windows installations or shelling out for VMWare or VirtualPC.

Unsurprisingly, Joe's revelation is causing quite a stir in the web development community. Matthew Haughey is asking why Microsoft didn't tell us about this themselves, Luke Redpath has released some funky coloured icons to distinguish between versions and Ryan Parman has taken the risky (from a bandwidth point of view) step of packaging the minimum files needed to run versions 5.01 and 5.5 up in to zip files. Amazingly, they're 2.92 MB and 3.25 MB respectively. I'm running them now and they seem to work just fine - major kudos to Joe for the discovery, and Ryan for making it so easy to take advantage of.

easytoggle and debugging in Safari

Simon Willison — 2003-11-06T01:28:49-00:00

I've been working on a new inobtrusive DHTML effect: easytoggle, which is an inobtrusive implementation of the common effect where links or tabs can be clicked to reveal part of a page while hiding the other parts. It's similar in some ways to the Multi part forms with Javascript technique.

The idea is pretty simple - all you need are a bunch of links and a bunch of elements that should be toggled by those links. When adding special behaviour to links it is always a good idea to ensure that they still link to something sensible, so that in user agents without Javascript support they still do something useful. In this case, it makes sense for them to act as anchor links that point to the elements with which they are associated. With easytoggle, all you need to do is define the links, point them at an element with an ID and assign them the class 'toggle'. The script does the rest of the work. For example, for a simple set with only two panels the markup would look something like this:

<ul>
 <li><a href="#p1" class="toggle">Panel 1</a></li>
 <li><a href="#p2" class="toggle">Panel 2</a></li>
</ul>

<p id="p1">This is panel 1</p>
<p id="p2">This is panel 2</p>

That's all it takes - the demo has a very slightly more complicated example.

I thought it was finished, until I tested it in Apple's Safari browser. In Safari, the worst possible thing happens. The initialisation code which hides all of the panels after the first one works fine. Unfortunately, the code that causes the links to change which panel is visible fails silently, leaving only the first panel accessible to users with that browser. That's a big problem.

It was at this point that I discovered that Safari's support for debugging Javascript sucks rocks. Firstly, the browser gives no indication that a bug has been encountered. I'm sure this is a deliberate usability decision, but it also means users who encounter a bug won't even know that there is a problem with the site. A quick Google led me to this page, which told me how to enable Javascript error reporting:

In a command line shell, execute defaults write com.apple.Safari IncludeDebugMenu 1 (apparently Safari Enhancer lets you do this from a GUI).
Reload Safari and check the "Log Javascript Exceptions" option in the newly enabled Debug menu.
Start Console.app, which lives in /Application/Utilities. Note that this is not the same thing as the command line console. This took me a few moments to figure out. Macs remain a strange new realm of discovery.
Javascript exceptions will now appear in the Console.app window.

Excellent - just what I needed to know. The entire error message I got when I clicked a link? (event handler):Undefined value. Gee, thanks a lot Safari. If anyone has any ideas how I can fix the script in Safari (or at the very least prevent it from being unusable) please leave me a note.

Update: Thanks to a tip from David Lindquist, the updated version of the script now works in Safari. It's a little bit uglier though.

Defeating browser incompatibilities

Simon Willison — 2003-10-29T20:03:18-00:00

Peter-Paul Koch has unveiled his new site, QuirksMode.org, which features over 150 pages of cross browser CSS and javascript tips and tricks. Five months in the making, there's just too much good stuff to link to individual pieces here so my best recommendation is to head on over there and spend some time browsing around. I'll make an exception though in linking to the Table of Contents script, which uses the DOM to create an additional navigation bar linking to each of the level 2 and 3 headers on a page. Another great example of the power of structural markup.