Data Should Be the Intel Outside
Version 1, changed by ScottMcmullan. 11/23/2005. Show version history
By ScottMcmullan at 11/23/2005 10:58AM Tags: REST, webservices, forms, opendata
Open Data
We've been talking about open data for a while now. So have lots of other folks, including Steve Mallet on DataLibre and Paul Kedrosky in his excellent post Data Should be the Intel Outside (via O'Reilly radar). The conversation refers to data in different contexts, e.g. the review data collected by Amazon or bookmark data at del.icio.us. Here's an excerpt from Paul's Data Should be the Intel Outside:
"Where should we be going? Call it "data as the Intel outside", where the innovation engine is how easily data can be recombined outside any one application. Turning things inside-out should be the Web 2.0 goal (or Web 3.0, as Steve Mallett puts it on his DataLibre site). We have open-source software messing up markets for shrink-wrap vendors of proprietary software, why shouldn't open-source data vendors mess up the market for would-be Web 2.0 vendors who are trying to Balkanize things by locking up data inside their own apps?"
But since the majority of JotSpot wikis aren't community sites like Amazon or del.icio.us, but rather passworded places where groups collaborate on the Web, our perspective on open data is slightly different. Our take, put simply, is:
It's your data, you put it in, you own it, and we want to make it as easy as possible for you to put it to any use of your choosing. Including packing your bags and taking your data with you if need be!
Pages are Just XML
Soooo great. What does that look like? I'm often saying "everything is a page in JotSpot", and "pages are just XML." But other than in the Dev Forums, I haven't given high-level tour on a Jot page's XML structure. Here goes.
Let's consider a very simple Jot page with some text on it: http://mysite.jot.com/WikiHome/MyTestPage
Here's the sample page's XML. By appending xml=pretty (aka prettyXML=1) to any page's query string, we get back the complete underlying XML for that page. This image below is an annotation of a view source done to: http://mysite.jot.com/WikiHome/MyTestPage?xml=pretty
Add Your Own Structure
It's all there, nothing held back. But what about structured data? One of JotSpot's strengths is the ease with which you can extend a page with your own structure. That structure opens new possibilities, as you can then edit/view/search/cross-reference your info in a ton of new ways.
Adding structure to pages is the foundation for creating applications in JotSpot, and is done using forms. Here's a very simple form, defined in another page called FooterNotesForm. When applied to another page, it will add a URL text field and 2nd text area for comments at the bottom of a page.
<wiki:form name="FooterNotesForm"> <jot:field name="main/text" type="wiki" style="height: 40em; width: 60em" /> !Notes: Related URL: <wiki:field name="relatedUrl" type="string"/> <wiki:field name="comments" type="wiki" style="height: 10em; width: 60em"/> </wiki:form>
After applying this FooterNotesForm to MyTestPage, it will looks like this when I edit it -- note the new Related URL and notes fields:
After saving the page, here's what the underlying XML looks like, again using http://mysite.jot.com/WikiHome/MyTestPage?xml=pretty:
Notice how our FooterNotesForm form extended the page's XML with a new <FooterNotesForm> element. We now have new FooterNotesForm/relatedUrl and FooterNotesForm/comments properties to use and remix with other pages and information! Not to mention a nice user experience for entering info as a web form.
Open Data JotSpot Style
To recap, we took a standard Jot page, chose some names for some new properties (relatedUrl and comments) using a Jot form, and Jot did the right thing. It extended our page's XML with the names we chose, stores our prose as XHTML, and handles everything for us when viewing and editing the page.
All the info we put in is freely and losslessly available in XML via a RESTful URL. We also provide a 1-click export of the entire JotSpot as a zip file, or can get a page or collection of pages as an RSS feed or a Excel/CSV document. Just in case you're not a REST hacker. Really any format that can be derived from XML, which is just about anything.
This is the basis for our approach to open data and our REST API. We'd love to hear your thoughts.
Back to WikiHome
Comments (5)
ScottMcmullan said, 07/18/2005:
> Does it mean that without the '?xml=pretty' passing,
> the raw XML page will already be parsed into XHMTL?
> Why is that?
Yes, that's basically the final stage of processing done by the server. It transforms the underlying page structure and output from running any script on the page into the appropriate XHTML, so it all displays nicely on a browser.
> <node>, <content>, <data> tags are predefined by
> your parsing engine? if we write an XML with those tags,
> can it be parsed in XHTML? just curious :)
Yep, if you create such an XML document and then save it to your JotSpot, you've effectively created a new JotSpot page. To "save it", you can put this XML document with a bunch of others in a zip file and then POST it to the server using importZip, or use the new savePage function (to be released/documented with the next releases).
Here's where the documentation is (or will be): http://developer.jot.com/WikiHome/JotDoc/JotDocDevs/RestApi
zibin said, 07/20/2005:
Thanks Scott, nice explanation :)
guest said, 10/23/2005:
the prettyXML appended to the query string will dump the XML codes when used in IE. It doesn't happen in Mozilla Firefoz though. ;)
guest said, 10/23/2005:
i've checked out BlogRSSCode
and realized that it uses a <rss> tag instead of the <html> tag in their XML code. Why is this permissible? and how? and whats the encouragement to do so?
Attachments (5)
| File | By | Size | Attached | Ver. | ||
|---|---|---|---|---|---|---|
 |
normalPage.png | ScottMcmullan | 9K | 07/14/2005 | 1 |
|
 |
prettyXmlOrig.png | ScottMcmullan | 80K | 07/14/2005 | 1 |
|
 |
normalPageWithForm-top.png | ScottMcmullan | 13K | 07/14/2005 | 1 |
|
 |
normalPageWithForm-bottom.png | ScottMcmullan | 8K | 07/14/2005 | 1 |
|
 |
prettyXmlWithForm.png | ScottMcmullan | 121K | 07/14/2005 | 1 |
|
guest said, 07/15/2005:
Finally a lowdown on the system :)
I understand that the underlying structure is really XML pages. Add a query string such as ?xml=pretty and you get the whole XML structure. But why is it the if we don't have the extra GET parameter passing such as ?xml=pretty, we only get a whole bunch of <XHTML> page through view source?
Does it mean that without the '?xml=pretty' passing, the raw XML page will already be parsed into XHMTL?
Why is that?
<node>, <content>, <data> tags are predefined by your parsing engine? if we write an XML with those tags, can it be parsed in XHTML? just curious :)