Adding Ubuntu One support to your applications, an IRC lecture

This year, as last year, I did an IRC talk as part of Ubuntu Developer Week. This time, it was about adding Ubuntu One to your applications, and what that will bring to your users. (Once again my xchat plugin for IRC talks was massively helpful!

Introduction

  • Hi! I’m Stuart Langridge, from the Ubuntu One team, and this talk is called “Adding Ubuntu One support to your applications”.
  • You can ask questions at any time in #ubuntu-classroom-chat. Please write QUESTION at the beginning of your question so I can see it more easily. I’ll stop at the end of a section and answer some questions.
  • During this talk I’m going to explain the different ways your applications can take advantage of Ubuntu One…
  • …talk about why you’d want to do that…
  • …and give a couple of sneak previews of things that you can’t *quite* do yet but will be able to do for Lucid 10.04, so you can be ready!
  • Firstly, though, remember that “integrating with Ubuntu One” isn’t really the point; the point is to give features to your users that make their lives easier.
  • Now, obviously, using Ubuntu One will make you taller, it’ll make you more attractive to the appropriate sex, and it’ll make the sun shine more brightly, but remember that it’s all about providing things that your users want…
  • …and Ubuntu One gives you the infrastructure you need to do that. If it doesn’t, come talk to us!

How to do cool things with file synchronisation

  • OK, so, the first thing is: files.
  • As you know, Ubuntu One automatically synchronises files in your Ubuntu One folder to Ubuntu One itself in the cloud and to all your other machines.
  • This means that you can take advantage of automatic file backup just by having your application save things to the filesystem!
  • Imagine that you want attachments that you receive by email to be stored online and backed up so you can get at them later.
  • Most mail clients will already let you do this in some way. So, change the folder that these attachments are saved in to be an Ubuntu One folder and suddenly all your attachments are backed up online and available on every machine!
  • For example, there is a Thunderbird attachment called “Attachment Extractor” (https://addons.mozilla.org/en-US/thunderbird/addon/556)
  • Install it, select “Automatically extract attachments on email receipt” in Tools > AttachmentExtractor settings > Auto-Extract, and set the “Save Path for Attachments” to /home/YOU/Ubuntu One/Mail Attachments, and that’s all you need!
  • Similarly, if you want off-site backups of your computer, then you can install one of the many simple backup programs, like backintime (http://backintime.le-web.org/) and choose the backup folder to be inside your Ubuntu One folder.
  • With no extra effort you’ve got off-site online backups which you can get at from any of your machines or from the web.
  • As another example, imagine you’ve built yourself a digital picture frame out of an old laptop, which is a pretty cool project for your weekends. How do you get the photos on it?
  • Well, when I did this, I created a new Ubuntu One user for the picture frame, and then I shared a folder with that picture frame user through Ubuntu One.
  • The picture frame just did this: eog —slideshow “/home/pictureframe/Ubuntu One/Shared With Me/pictures from Stuart Langridge”
  • and then to add new pictures to the frame, I just copy them into the folder that I shared.
  • I could share that folder with anyone else, too, and allow them to write to it, and then they’d be able to add pictures to the frame too!
  • The way I’ll do this from 10.04 Lucid onwards is to use the new User Defined Folders feature, where any folder, anywhere, can be designated as an Ubuntu One synced folder
  • and you can elect to have some folders but not others synced to particular machines
  • So I’d create my “digital picture frame” folder in my home folder (or wherever), and then mark it as a UDF.
  • Then on the picture frame itself, I’d sign into Ubuntu One, and say that I only want to subscribe to that particular UDF and not any of the others.
  • so then files that I put into “digital picture frame” on my laptop will be synced to the picture frame
  • and the picture frame just displays all the files in that folder
  • ta daaah, it Just Works.

Publishing files publically

  • A much-requested feature which will be available in Lucid is “public files”, the ability to publish an Ubuntu One-synchronised file to a public URL so anyone can get at it.
  • This is useful for the user directly; they can easily make a file available to the world.
  • It’s also really useful for applications, because you can now build one-click publish-to-the-world directly into your apps.
  • Imagine that your app is Shutter, which is like the standard Gnome Screenshot tool on steroids.
  • If you install Shutter and then hit the Print Screen button, it’ll take a picture of your screen.
  • One of the main reasons people take screenshots of their screen is to show them to other people, which makes this a perfect case for publishing the screenshot to a URL so anyone can see it.
  • So, in Shutter (or your similar app), provide a “Publish with Ubuntu One” button.
  • That button should do the following:
    1. Save the file into the user’s Ubuntu One folder (just save, as normal)
    1. Call the publishing API
  • The publishing API will be D-Bus; I can’t confirm exactly how it will work, yet, because the team are still constructing it, but keep your eyes open for more details!
  • This shows a basic principle of Ubuntu One’s design; pretty much anything you can do explicitly, like publish a file, or share a file, can also be done using the D-Bus API from applications.
  • I’ll answer some questions here about using Ubuntu One file sync in your apps.

Questions

  • kklimonda|lernid QUESTION: Is there a full GObject U1 API for storing data, settings etc. or do we have to make use of \~/Ubuntu One/ for now?
  • Data and settings are best stored in desktopcouch if you want to take advantage of all that lovely synchronization goodness; there are benefits to doing it that way, and I’ll talk about desktopcouch in a bit.
  • But you can certainly store settings files in \~/Ubuntu One if you want to
  • and obviously actual data *content* that your apps work on, your documents and so on, are stored in \~/Ubuntu One
  • mhall119|work QUESTION: in 10.04 you will be allowed to have multiple sync folders outside of \~/Ubuntu One/ ?
  • mhall119|work, yep, you sure will. UDFs (user defined folders) are a big feature that’ll land in 10.04.
  • strycore89 QUESTION : will we be able to share single files and not whole directories ?
  • No. Sharing is done folder-by-folder, and we’re not planning on changing that.
  • mhall119|work QUESTION: right now the local folder names match the online folder names. With the ability to have multiple sync folders, how are naming conflicts resolved?
  • I’m not sure what you mean here by “naming conflicts”; maybe we can pick that point up after the session, or later on?
  • foobob QUESTION: are you thinking of giving some GB more to the free plan?
  • I don’t believe we are at the moment, no.
  • yltsrc QUESTION: will be ready symlincs in U1?
  • yltsrc, UDFs, being able to mark any folder as Ubuntu One synchronized, takes the place of symlinks. If you use, er, some alternative to Ubuntu One and you’re used to putting a symlink in, say, \~/SyncedFolder which points at \~/SomeOtherFolder to get SomeOtherFolder synced, you don’t need to do that in Ubuntu One; you just mark SomeOtherFolder as a UDF and it will be synced for you.
  • lantash49 QUESTION: Will there be more detailed specification w.r.t storing settings in \~/Ubuntu One? Storing them there directly seems to conflict with the XDG directory specs.
  • lantash49, one obvious approach to this is to mark XDG_CONFIG_DIR and XDG_DATA_DIR as UDFs, so they’re synchronized across your computers.
  • yltsrc QUESTION: i want participate in testing U1, how can I do it?
  • Drop into #ubuntuone and talk to the team about what’s going on and how you can help; we’ve had some really helpful community members already, like rtgz, who’s done tons of stuff
  • taj QUESTION: Are there any plans to make U1 cross-platform?
  • There’s a sprint at PyCon US to port Ubuntu One file sharing to Windows:
  • and mandel, another incredible community member, is currently porting desktopcouch to Windows: http://www.themacaque.com/?p=372
  • both of those projects are great, and if you can help that’d be wonderful
  • mhall119|work QUESTION: If I put a symlink inside of a UDF, that points elsewhere, does U1 sync that as a link, or as a copy of what is linked to?
  • it’s synced as a symlink.

Desktop Couch

  • The second leg of Ubuntu One for developers is desktopcouch, the CouchDB on your desktop.
  • If you missed my last Ubuntu Developer Week talk about the basics of desktopcouch and what it’s for, you can read it at http://www.kryogenix.org/days/2009/09/03/desktop-couch-irc-talk
  • In fact, you can read all the desktopcouch documentation at http://www.freedesktop.org/wiki/Specifications/desktopcouch/Documentation/, and you should do so :-)
  • The bit that’s most likely to be relevant here is the “I am a developer, and I want the applications I write to store data in and work with desktopcouch” section. It contains much that is relevant to this talk!
  • http://arstechnica.com/open-source/guides/2009/12/code-tutorial-make-your-application-sync-with-ubuntu-one.ars/1 is a really great guide, written by segphault, of how to support cloud synchronization of data in your applications.
  • Essentially, if you use desktopcouch to store your application’s data, instead of using sqlite or flat files, then the data will be synchronised between all your machines *without you having to do anything*.
  • So your app will work the same on all your machines with no effort.
  • To store a record in desktopcouch, simply do:
  • from desktopcouch.records.server import CouchDatabase >>> from desktopcouch.records.record import Record >>> db = CouchDatabase(‘testing’, create=True) >>> r = Record({‘key’:’value’}, record_type=’http://example.com/testrecord’) >>> record_id = db.put_record(r)

  • and to get it back, by record ID:

  • fetched = db.get_record(record_id) >>> print fetched[‘key’] ‘value’

  • The Quickly project, for writing Ubuntu apps, have created “widgets” that use desktopcouch without you having to think about it. CouchGrid, for example:

  • from desktopcouch.records.couchgrid import CouchGrid >>> keys=[“key”,”another key”] # to label the columns >>> couchgrid = CouchGrid(db_name, record_type=record_type, keys=keys) >>> mywindow.pack_start(couchgrid)

  • and then you have a CouchGrid, which is a data table where all the data is automatically retrieved from desktopcouch and stored back into desktopcouch when changed

  • http://www.youtube.com/user/calorielookup#p/a/u/0/Vwr5Xw5ZrIE has a video demonstrating couchgrids
  • You don’t have to know anything about desktopcouch at all for this! Just use the widget and your data will be saved to the database and synced to your different machines with no effort or code on your part at all.
  • https://wiki.ubuntu.com/Quickly/Snippets has a few more examples of using desktopcouch from Python and from Quickly apps.
  • So, cloud-enable your apps! Let’s see my song ratings in Rhythmbox or Banshee or Amarok or Exaile or Quod Libet appear on all of my machines. Let’s see my Chrome bookmarks appear everywhere, like my Firefox bookmarks already do.
  • Remember, too, that apps can share data. If you want to store song ratings for Amarok in desktopcouch, don’t make that storage Amarok-specific; that way, if I decide to switch to Quod Libet I don’t have to re-rate all my songs!
  • I’m happy to answer questions about sharing of data between apps if you have them.
  • Also, I’ll stop now to take some questions about desktopcouch generally.

Questions

  • andypiper QUESTION: what is Quickly….?!
  • Quickly is an app that helps you make Ubuntu apps really…quickly
  • it takes care of putting together the framework of an application for you, and it makes a deb package for you and uploads the code to launchpad, etc. All the boring stuff that you don’t care about when you just want to get your app done
  • the next session, by didrocks, is about quickly, so I’d stick around for that :-)
  • https://wiki.ubuntu.com/Quickly has more
  • yltsrc QUESTION: is available any standard for database name? And how resolve conflicts with different application?
  • yltsrc, there isn’t a standard, as such, for database names, because what the desktopcouch project is trying to avoid is laying down lots of standards that you Must Comply With
  • we’re a “rough consensus and running code” sort of group :-)
  • the best place to have discussions about that sort of thing is at http://groups.google.com/group/desktop-couchdb which is the desktopcouch mailing list
  • desktopcouch records are deliberately set up to have two sections, a “main section” and an “application specific” section
  • so the idea is that apps co-operate on the main section and store app-specific data in the app-specific section
  • for example, Evolution contacts sync in Ubuntu One is done by Evolution storing your contact data in desktopcouch
  • but the format for a contact has been defined
  • and the contacts are stored in a database in desktopcouch called “contacts”
  • so Thunderbird, for example, could *also* sync its contacts to that same database, in the same format
  • and Evolution stores Evo-specific data (like, say, an internal ID) for a contact record in the app-specific part of the contact record
  • and Thunderbird would do the same
  • what this means is that if you decide to switch from Evolution to Thunderbird, you’ve still got your addressbook! You don’t have to export it and re-import it every time you move between applications
  • yltsrc QUESTION: U1 require python (2.6). Is any plans to run it on python3?
  • I don’t believe there is, at the moment, no.
  • b1ackcr0w QUESTION: SpiderOak is said to be going Open Source soon, are there oppertunities to integrate their services with U1?
  • There could be, certainly. I haven’t looked into that in detail myself, but I’ll be sure and bring it up!
  • mhall119|work QUESTION: I want to hack on the server-part of U1, short of becoming a Canonical employee, what can I do?
  • At the moment, you can’t, I’m afraid. The source code for the server part of file synchronization is closed.
  • The source code for the server part of desktopcouch is CouchDB, though, and work on that is perfectly doable.
  • We worked closely with the CouchDB upstream project to make sure that all the changes we needed to use CouchDB in Ubuntu One were part of the main CouchDB trunk.
  • yltsrc Is avalably any FEATURE ROADMAP for U1?
  • that’s…sorta what this talk is :) I can see how it might be useful to have a more formal statement of “what’s coming up”, though. I’ll bring that up with the team.
  • yltsrc QUESTION: is U1 uses dbus? and how i can extend my application to use it?
  • Yes. The syncdaemon, which is the bit that handles file synchronization, exposes D-Bus commands for almost everything it does.

Web apps and desktop apps talking via Ubuntu One

  • Since desktopcouch is a CouchDB, and since your data is synchronised not only to your other machines but up to the cloud as well, this opens up the intriguing possibility that you can build a web app that can share data with your desktop app.
  • Ubuntu One’s CouchDB uses OAuth for authentication.
  • This means that your web app can ask permission from the user to read your desktop app’s CouchDB data directly from Ubuntu One.
  • So the data that appears in your desktop app can also be available to that user on the web.
  • This is a pretty new area, so nobody’s using it yet, but it’s possible.
  • You use OpenID to log the user in to your web app using Ubuntu One — you may have seen “log in with Twitter” buttons, or “Facebook Connect”, which work on the same principle.
  • Once that’s done, your web app can then request permission to read the user’s CouchDB database to get at the data that your *desktop* app stored there.
  • So, if we think about the song ratings example above, you can build a web application that can show all the song ratings that the user has entered and allow them to be edited.
  • Your users can then manage their song ratings or playlists or whatever else you’ve stored directly from a web interface if they’re not at their actual computer.
  • Sharing data between web apps and desktop apps is quite hard, and not many people are doing it, so this is a chance to offer a feature that no-one else has!
  • DesktopCouch plus web apps give you the ability to share that data without you having to do any extra work.
  • This also gives you the ability to offer mobile apps as a web interface, so people can work with their data from their smartphone and see their work reflected on their desktop when they get back to it.
  • Are there any questions about how linking up desktop apps and web apps would work?

Questions

  • mhall119|work QUESTION: how does it handle when a file is edited on2 different computers before syncing?
  • If that happens, you’ll get a .u1-conflict file showing you that there was a conflict, for file synchronization.
  • In desktopcouch, there will be a conflict record for the data.
  • One of the things that we have on our Big List Of Stuff To Do is some sort of graphical conflict management to make dealing with these situations even easier, but that won’t be done for 10.04.
  • So at the moment, handling data conflicts is done by the applications that use the data; the applications have a better idea about how to merge/overwrite/handle conflicts than Ubuntu One does most of the time anyway, because Ubuntu One just moves the data around, it doesn’t always understand it.
  • qense (late) QUESTION: When you define a UDF outside the Ubuntu One directory, where will it be synced to when you enable/install Ubuntu One on a different computer?
  • It’ll be synced to the same path, so if you mark \~/myfolder as a UDF on computer 1, it’ll appear as \~/myfolder on computer 2.
  • duanedesign QUESTION: Where can I find documentation on the U1 API
  • the desktopcouch API is defined (mostly) in the desktopcouch documentation at http://www.freedesktop.org/wiki/Specifications/desktopcouch/Documentation
  • the file sync API isn’t defined anywhere yet, I don’t believe, because we haven’t had a chance to do it, so it’s reading the source, I’m afraid. https://wiki.ubuntu.com/UbuntuOne has some details, though.
  • emoreno QUESTION: Is there any way to use a Private Cloud rather than U1 with the same API
  • Not at the moment. The protocol that the syncdaemon uses to talk to the cloud isn’t secret, though, so it would be possible to implement a private cloud.
  • mhall119|work QUESTION: does U1 offer any kind of versioning on it’s data, like the ability to retrieve an older version?
  • Not at the moment, no. We’ve talked about that, but it isn’t available now.
  • OK, that’s a brief tour of what services Ubuntu One can bring to your apps. You can make the files your app uses be synchronised, you can publish those files publically, you can store data from your apps that will be available everywhere, and you can share that data with web applications.
  • nealmcb QUESTION: Re the question on working on the U1 server end
    • Elliot Murphy recently talked about your open file storage protocol, noted his “build no silos” policy for U1, and talked about e.g. Tomboy integration. Can you expand on the places that people can work on servers that integrate with U1 protocols?
  • desktopcouch is a CouchDB, so it can replicate data with any CouchDB, including other desktopcouches on your LAN — see the desktopcouch-pair tool for this, in the desktopcouch-tools package — and so doesn’t even need a server at all
  • Tomboy notes sync was done by us implementing the Snowy API — Snowy is a Django server to which you can sync your notes, designed by the Tomboy team, and they’re looking at doing their own deployment of it
  • qense QUESTION: Any change there will be a small collection of snippets showcasing webintegration of Ubuntu One?
  • if someone does that I’ll love them forever.
  • I’d certainly like to, it’s just finding the time :-)
  • OK, if you have other questions, feel free to ping me, or chat on #ubuntuone!

More in the discussion (powered by webmentions)

  • (no mentions, yet.)