[9:31] Zha Ewry: Plenty of room for more aves, but I may have to rez more chairs
[9:31] SignpostMarv Martin: applications for SL's 5th Birthday end on the 25th of May, though I know that may be a bit short-notice to make adecision as to what would go in it :-P
[9:31] Zha Ewry: Donovan, the little beanbag facinfg us is yours to grab when you're ready ;-)
[9:37] Donovan Linden: Ok. So just for some background, the SL architecture, when it was pure UDP, allowed for sending messages to the client that the server initiates, without the client making a request
[9:37] Zha Ewry: I think there are enough benbags in the outer row. I bet some are facing the wrong way. if so, sing out and I'll turn you areuond
[9:38] Tao Takashi: will try not to repeat himself too much today ;-)
[9:38] Donovan Linden: When we did the Message Liberation project which allows arbitrary messages to be moved over tcp instead of udp, we needed the same semantics
[9:38] Zero Linden: oh - these are the new beanbags - shift-arrows change your pose!
[9:40] Donovan Linden: So, we used a technique called "The long poll" where the client is always making a request to the server, but the server does not respond until there is an event to send to the client
[9:43] Xugu Madison: I take it that opening multiple connections from the client to the server wouldn't really be a solution because of the amount of resources (sockets) being tied up?
[9:43] Donovan Linden: The comet long poll technique is great because it requires only a normal HTTP client which should be available in almost any programming environment
[9:43] Gareth Ellison: breaks the HTTP RFC too - which i think we agreed was a bad thing(TM)
[9:47] Zero Linden: ah ... okay keep-alive connection limit, gotcha
[9:47] Donovan Linden: pipelining is different than keepalive, it allows the client to make another request in the same socket before the server has responded to the first one
[9:48] Donovan Linden: For people that don't believe comet works, gmail uses comet. google docs uses comet. meebo uses comet.
[9:48] Saijanai Kuhn: and SL uses a comet-like thing also
[9:48] Gareth Ellison: i doubt we have any heretics here donovan ;)
[9:48] Donovan Linden: But, since we have our own custom client in the Second Life viewer, we don't have to jump through as many hoops as normal web browsers do.
[9:49] Tao Takashi: everybody should use an SL client to browse the web
[9:49] Donovan Linden: If we're willing to use a custom protocol, we can set things up so the server can push messages over a tcp socket without the client requesting it
[9:49] Gareth Ellison: is morgaine here? she had an interesting thought on just using HTTP to setup the connection and then scrapping HTTP semantics once the socket is open
[9:49] Jrift Sol: that sounds like it might cause issues with firewalls though
[9:52] Donovan Linden: The Upgrade and 101 Switching Protocols semantics are part of HTTP/1.1, so anything that follows the spec should be able to handle it
[9:53] Donovan Linden: Once the server agrees to Upgrade, the server would then start using the socket as if it were a client, and the client would use the socket as if it were a server
[9:53] Donovan Linden: However, we can also add a fallback for safety to this design.
[9:53] Zero Linden: / believes that most consumer level firewalls don't look at the wire protocol (though a few do to block URLs, but that is all) -- whereas firewalls sold to corp. IT often have lots of wire inspection - often to support the high-$
[9:54] Donovan Linden: We can change the Reverse HTTP spec to require that the server send the client a setup message immediately after Upgrade
[9:54] Gareth Ellison: consumer level firewalls come in a few flavours - basic filters/ACLs or proxies with censorware
[9:54] Code Tracer: yeah zero, i know couple of firewalls do content security such as check point... etc
[9:54] Gareth Ellison: hands up anyone who has attended a high school in the last decade and hasn't experienced the censorware firewalls
[9:54] Jrift Sol: i've opened a can of worms here. sorry :)
[9:54] Donovan Linden: If the client does not recieve the message within a timeout, the client can make the request again without the upgrade header, and fall back to the COMET-style protocol.
[9:55] Code Tracer: well actually, costumer, anti-vires applications do sniff ur connection against backdoors and trojans
[9:55] Rex Cronon: try going to a public library gareth:)
[9:55] Code Tracer: they might block reverse http thing
[9:55] Jrift Sol: my university certainly inspects packets. access to the sl website is even blocked there
[9:55] Donovan Linden: There are also other options such as multipart streaming COMET which is another way to optimize the latency without falling back all the way to one client request per server push
[9:56] Gareth Ellison: (offtopic in the extreme) my own site is routinely blocked by local libraries, unblocked when i complain and later blocked again in cycles
[9:56] Gareth Ellison: donovan - with long poll, can the socket remain idle for a horrendously long amount of time?
[9:56] Donovan Linden: however, I think what is on the Reverse HTTP page, amended with some information about what the client should do after a timeout if the server wants to Upgrade but a proxy or firewall won't let the real PTTH protocol through
[9:56] Gareth Ellison: or do we need keep-alive requests
[9:57] Donovan Linden: Gareth: With the long poll it's generally a good idea to time out after 25-30 seconds to avoid bad NAT implementations which have short idle timeouts
[9:57] Donovan Linden: keep-alive is orthogonal to the long poll; it's a good idea but it's not required
[9:57] Zero Linden: / thinks current implementation in viewer is timinig out at 30 seconds
[9:58] Donovan Linden: the current second life viewer makes a new HTTP/1.0 request every time
[9:58] Zha Ewry: / One thing to do, with things like this, is actually build a tiny test utility and pass it around widely
[9:58] Donovan Linden: zero: yes, that is because squid is killing the connection after 30 seconds of idle
[9:59] Gareth Ellison: an idea - put such a test util up on the secondlife downloads page
[9:59] Code Tracer: donovan; using this reverse http thing is a big over head... why dont u open a simple socket and keep it alive, instead of using all these http servers. proxies..
[9:59] Gareth Ellison: get the unwashed masses to get some realworld data
[10:00] Gareth Ellison: do we really need the HTTP semantics if we're using it for streaming data
[10:00] Saijanai Kuhn: Libraries are well tested for http
[10:00] Morgaine Dinova: Code: don't talk heresy. Everything has to be HTTP, you know that :-)))
[10:00] Zha Ewry: Because then you get reo redo the full semantics and have your own
[10:00] Zero Linden: the reasons for using HTTP, even beyond libraries
[10:00] Xugu Madison: Saijanai, but if you're sending a message, that's hardly rocket science...
[10:00] Zero Linden: is that we'd have to define wire protocol for almost everything there: status codes,
[10:00] Donovan Linden: the disadvantages of using a custom protocol are even worse, because we'dThe major advantage of Reverse HTTP is that the implementation can be incredibly simple using existing HTTP servers and clients
[10:00] Gareth Ellison: as i see it, we're currently hacking around some defiencies in HTTP - which works great for one-off messages or file transfers, but crap for event streams
[10:02] Donovan Linden: those two lines at the bottom of the example actually make a requet from the server to the client
[10:02] Saijanai Kuhn: Tao its so simple its confusng.
[10:02] Gareth Ellison: hmm, coming from a background where XML-RPC rules the world (any other current or former opensim guys here?) i can see why HTTP is nice for some kinds of RPC
[10:03] Donovan Linden: it would be better to break the script into client and server peices, so it's clearer and can be used to test real firewalls and proxies
[10:03] Zero Linden: +1 donovan (I was just going to say tha)
[10:03] Gareth Ellison: has LLSD over REST for RPC stuff
[10:03] Tao Takashi: it's maybe small, not sure it's simple ;-)
[10:04] Donovan Linden: Another disadvantage of using a protocol other than HTTP as the push protocol is that we'd have to design and implement it, and everyone would have to implement it in each language they wanted to use
[10:04] Gareth Ellison: if LL ever patent LLSD somehow i'm completely and utterly screwed
[10:04] Donovan Linden: most languages already have http clients and servers that can be adapted to this
[10:04] Gareth Ellison: ok, settled - HTTP is the winner
[10:04] Tao Takashi: many people would feel more comfortable implementing OGP if they knew it's using somethign they know, which is HTTP
[10:04] Tao Takashi: even if it's HTTP with a twist
[10:05] Zero Linden: Garth - I think LLSD has been documented publically for over a year -- which would mean we couldn't patent it
[10:05] Morgaine Dinova: Zero: a lot of those things you mentioned are severe over-complication. All you really need is a TCP stream and a method of serializing. You don't need retry semantic in the envelope when all you're doing is transferring data structures which themselves contain error indicators.
[10:05] Tao Takashi: so basically what they need to understand is Donovan's code snippet then ;-)
[10:05] Gareth Ellison: zero - true, but stranger things have happened
[10:06] Tao Takashi: even if it might have slight advantages (but we might not know the disadvantages yet)
[10:06] Donovan Linden: oh, one last thing about Reverse HTTP -- since it is going to be going over HTTPS, I think that seals the deal for all the worries about proxies and firewalls
[10:06] Zero Linden: I strongly disagree - that is practically what the current UDP system is - though even there, there is framing, error indicators, non-delivery, header info....
[10:06] Zero Linden: and without those things, extension is very very difficult
[10:07] Donovan Linden: proxies and firewalls should work just fine since they already have to allow https traffic without knowing about the body
[10:07] Tao Takashi: the whole UDP format stuff prevented me from doing a python libsl
[10:07] Jrift Sol: donovan, you mentioned multi-part streaming comet earlier. So far, that sounds like a safe (as far as packet inspecting firewalls go) method, and not too different from what's currently in use, and goes a long way to solving the latency where there's a lot of updates. What problems do you perceive with that?
[10:07] Zero Linden: This is true - once you've negotiated an end-to-end SSL connection - the firewalls can't look inside
[10:07] Morgaine Dinova: Donovan: but you're still thinking about replies! See, you still have HTTP on the mind. I'm thinking about a data stream. HTTP is just a means to an end, for getting that data across. And 90% of it is unnecessary.
[10:07] Tao Takashi: because nothing was/is documented, UDP in general is more complicated to implement (esp. if you add TCP functionality to it) etc.
[10:08] Tao Takashi: but for me it started with the packet format that the hashes used back then did never to be right
[10:08] Donovan Linden: jrift: that requires defining the semantics for replies, replies can come out of order, and you'd have to decide what to do about dropped messages
[10:08] Tao Takashi: and there was also no library you could use to help it
[10:08] Saijanai Kuhn: well, the packets are all documented in the SL source code. I've been using them
[10:08] Tao Takashi: "documented in source code".. well :)
[10:09] Tao Takashi: but back then there als have been these hashes and no explanation of how they got computed
[10:09] Tao Takashi: anyway, every custom format makes implementations harder
[10:09] Tao Takashi: and we want adoption so we should go with known standards where possible
[10:09] Donovan Linden: also, it seems like a huge advantage to have all messages be REST (except for a very few messages which will stay UDP and not have replies)
[10:09] Gareth Ellison: i really want to clarify - we have HTTP transport that is unused in the viewer already, right?
[10:09] Donovan Linden: Gareth: it's not unused, it's heavily used
[10:09] Donovan Linden: every udp message that now goes over tcp uses it
[10:10] Saijanai Kuhn: ah, well, libsl uses its own hash. I just index off of the packet ID num
[10:10] Gareth Ellison: well, yeah - but a whole load still goes over UDP
[10:10] Gareth Ellison: it should be possible to shove it all over HTTP, right?
[10:10] Tao Takashi: Sai: It might have changed, I haven't looked into that for 1.5 years ;-)
[10:10] Donovan Linden: and some of those messages will stay udp. any message which is time-sensitive and an old message can be dropped if a new message has been received should go over udp.
[10:10] Saijanai Kuhn: OK, only started doing it 6 months ago, so major changes I'm sure
[10:11] Zha Ewry: Well, lettign UDP drop messages, puts you at the mercy of it's total lack of semantic understanding of what to drop, we've talked about that
[10:11] Morgaine Dinova: Donovan: the only reason you're making those HTTP calls is to get data back (other than the side effect of changes on the server). If you have another means of getting that data across then the two things can be separated: (i) the perform-action side effect for the server, and (ii) the checking of data (which may be a response) on the client side. No longer any need to couple the two things.
[10:11] Tao Takashi: it definitely makes implementation easier if you don't have to worry about housekeeping
[10:12] Jn Uriza: surely your now talking about encapsultating a higher level protocol or logic within HTTP so why bother, why not skip the whole HTTP overhead altogether? or maybe I'm missing the guts of this
[10:12] Donovan Linden: if you want an ack, you should use rest semantics
[10:12] Zero Linden: Right - pushing all that coordination into the higher levels means that each and every sub-system has to do that work
[10:12] Zero Linden: which is what is going on in our code base now
[10:12] Zero Linden: it is a tremendous duplication of effort
[10:12] Donovan Linden: Jn: because we want request/response semantics because we want atomic messages
[10:12] Zero Linden: becuase the vast majority of interactions in the protocol are request/response
[10:12] Tao Takashi: goes back looking into Donovan's script :)
[10:22] Saijanai Kuhn: thinks that test harness will be first use of reverse http: command sending via post from control script/web page to test harness
[10:22] Zha Ewry: I'm a huge fan of eatting our own dogfood
[10:23] Zha Ewry: But.. it does make it necessary to be very good about posting transcripts and publisziing meetinfgs
[10:23] Tao Takashi: well, I am not against meetings in here but IMHO it would also be nice to do things more via mailing lists like inbetween meetings
[10:23] Tao Takashi: and transcripts honestly are a pain to read
[10:23] Gareth Ellison: i'd prefer IRC personally - way way easier to discuss things without the avatars
[10:28] Morgaine Dinova: Tao: want to hold a meeting sometime specifically about that code, and where it can be taken?
[10:28] Tao Takashi: sure, maybe I can also make a little screencast
[10:28] Zha Ewry: Anyone who wants to hold those, is welcome to use this space
[10:29] Tao Takashi: I am just not sure when, this week is a bit full (and there's a holiday) and next week I have to give a Plone training and I am speaking about OGP at the Metaverse08
[10:30] Gareth Ellison: why do these sexy conferences never happen in north wales? :(
[10:30] Tao Takashi: but we have at least Linden Lab attendance :)
[10:30] Tao Takashi: but not for talking about AWG/OGP unfortunately
[10:31] Tao Takashi: and if somebody wants to come to EuroPython, registration is open and I will do a sprint about OGP if somebody shows up who is interested
[10:32] Tao Takashi: if you want to give a talk, deadline is in 2 days ;-)
[10:32] Tao Takashi: (for the topic not the contents)
[10:32] Python Morales: Tao, browsed thru your ogp code now .. not much meat yet (or did i mess something?) but interesting .. and i guess zope.interfaces may well be good for this kind of stuff
[10:32] Tao Takashi: I will also give a talk about pyogp there
[10:34] Python Morales: zope.interfaces kinda gives you a way to make interface definitions in py, like you can do in c# or java with interface declarations
[10:34] Gareth Ellison: has roughly 2GB per day of new data shoved into MySQL
[10:35] Gareth Ellison: it hasn't crashed down painfully yet
[10:43] Enus Linden: Q and I are going to chat about an hq repo vs LL svn fwiw. we can talk about Google Code too... and it has been suggested that Apache v2 license would be appropriate for this effort.
[10:43] Morgaine Dinova: I'm surprised that wikis don't get used for collaborative design and coding. They're almost ideal, with an attached SVN. Far better than MLs, which are hard to focus.
[10:43] Gareth Ellison: svn is pretty much the defacto standard these days
[10:44] Tao Takashi: I am not sure, wiki's also tend to grow into various direction and then it's hard to find stuff again
[10:44] Gareth Ellison: though i can remember when i called a friend a heretic because he'd recently imported his CVS code into SVN
[10:44] Donovan Linden: hg is a million times better than svn
[10:45] Gareth Ellison: SVN == largest deployment, everyone and his dog has a client
[10:45] Tao Takashi: I don't care what it is, as long as a) there is something b) the decision process does not take 2 months ;-)
[10:46] Tao Takashi: I tried it the first time today to get the eventlet code
[10:46] Python Morales: gareth, svn is basically cvs with basic annoyances fixed - but the same model. distributed systems like hq and bzr and darcs etc. are a new generation in design and have benefits. i think it comes down to needs .. some projects need distributed more, some are fine with centralized
[10:47] Morgaine Dinova: I don't think the key problem here is *which* svn.
[10:47] Enus Linden: I spoke with Robla about licenses, as he is far far more familiar with them that I, and he suggested Apache v2. Zha had mentioned the same... [14]
[10:47] Gareth Ellison: hmm, i'll admit that my current scheme of mounting all my SVN repos from central location via sshfs is a bit of a pain
[10:47] Tao Takashi: then I will wait with my pypi upgrade until that's around
[10:50] Zha Ewry: Apache v2 is nicely compatible with most of what we're doing
[10:50] Tao Takashi: Zha: what are the "features"? :)
[10:50] Gareth Ellison: from a brief reading it seems to be a copyleft - am i correct?
[10:50] Zha Ewry: Its BSD with a bit of the patent torpedo loophole closed
[10:50] Morgaine Dinova: Sold. Get hg installed. And then stick the client sources in there too, and let the community make its own client, which you build and release at same time as yours :-)
[10:51] Tao Takashi: CG Linden mentioned that this is the plan
[10:51] Donovan Linden: the only problem with hg is hosting
[10:57] Gareth Ellison: nothing like sitting in a dark room with a python REPL on my terminal, an unhealthy pile of cans of red bull and said REPL running on a live server daemon with 10 users connected