I found Mike Davies Breaking the Web with hash-bangs to be a frustrating article. Davies ends with a genuinely useful proposal, but, before he gets there, he goes through a lot of explanation that makes it look like he doesn’t understand the problem that #! attempts to solve. And he suggests (in one of the comments) using an HTML5 API, which would replace the (extraordinarily bad) flaw, that Gawker isn’t indexed, by the (even worse) flaw, that its page links wouldn’t work in the majority (by market penetration) of consumer browsers.
The tl;dir is:
- It’s traditionally been impossible to get both a single-page interface (SPI) and deep linking.
- Google works around this by promising to index fragment ids that start with ‘!’. Voila! A web developer can create an SPI that Google can index.
- Gawker implements to this spec.
- However…only Google Search implements this spec. Gawker is now invisible to other search engines / crawlers — even Google News!
This is a great solution embedded within an article that blows its credibility too early.
 For example: “Why? I don’t know. Twitter’s response when faced with this question on launching ‘New Twitter’ is that Google can index individual tweets. True, but they could do that in the previous proper URL structure before too, with much less overhead.” But the previous URL structure doesn’t support the single-page interface that Twitter and Gawker have implemented.
 An SPI requires that the URI (before the #) remain the same (otherwise the browser, prior to HTML5 pushState(), forced a page reload); deep linking requires a unique URI (not using the fragment id to distinguish URLs) for each resource. Search engines don’t index fragment ids.