Logs for jdev

Show join/part/nick changes:

[00:45:47] * hawke left the chat.
[00:58:06] * darkrain_ joined the chat.
[01:06:42] * waqas joined the chat.
[01:32:22] * darkrain_ left the chat.
[01:35:56] * jameschurchman joined the chat.
[01:36:51] * jameschurchman left the chat.
[01:50:02] * evilotto left the chat.
[01:53:29] * jameschurchman joined the chat.
[02:00:27] * jameschurchman left the chat.
[02:17:55] * naw left the chat.
[02:19:30] * darkrain left the chat.
[02:22:00] * D2 joined the chat.
[02:25:12] * louiz’ left the chat.
[02:25:16] * louiz’ joined the chat.
[02:46:31] * waqas left the chat.
[02:58:24] * Florob left the chat.
[02:58:27] * Florob joined the chat.
[03:05:24] * Florob left the chat.
[03:05:30] * Florob joined the chat.
[03:13:55] * jcea left the chat.
[03:32:46] * Florob left the chat.
[04:05:41] * louiz’ left the chat.
[04:05:58] * louiz’ joined the chat.
[04:33:20] * waqas joined the chat.
[04:38:52] * louiz’ left the chat.
[04:39:10] * louiz’ joined the chat.
[05:10:55] * louiz’ left the chat.
[05:11:00] * louiz’ joined the chat.
[05:20:26] * D2 left the chat.
[05:31:02] * jb joined the chat.
[05:31:44] <jb> Hello, I'm a new user
[05:32:14] <waqas> Hello
[05:32:48] <jb> Hi, I want to get started with a jabber chat server, can you help?
[05:33:26] <waqas> Here's a list of servers: http://xmpp.org/xmpp-software/servers/
[05:34:23] <waqas> What are your requirements?
[05:34:32] <jb> Thank you, I was aware of the xmpp.org site
[05:35:23] <jb> Can the xmpp servers run on a linux shared server?
[05:36:26] <waqas> If the shared server allows running persistent processes, and listening on port 5222 and 5269, yes.
[05:37:40] <jb> Are the xmpp servers essentially socket servers that must run from the physical server's CLI, with a root account?
[05:38:55] <waqas> They are socket servers, but socket servers can run without root just fine. I know that Prosody (which I work on) runs fine in limited accounts, as long as it's allowed to listen on those ports.
[05:39:47] <waqas> (in fact Prosody complains when running from root, since it doesn't need to)
[05:40:51] <jb> So the primary requirement is the ability to execute programs on the physical server without a timeout limit?
[05:41:24] <waqas> Yep. At least for Prosody. I can't say that for sure about the other servers.
[05:42:28] <jb> I want to operate a chat service, but question whether I need to run a dedicated xmpp server.
[05:43:40] <waqas> How many simultaneously online users do you expect?
[05:44:26] <jb> Don't know, but let's say up to 10,000 concurrent users, as an example
[05:46:06] <waqas> You may need a decently capable server for that. you can run your own, or try one of the hosting options available.
[05:46:37] <jb> What are the hosting options?
[05:47:48] <waqas> http://wiki.xmpp.org/web/Jabber_Hosting_Possibilities
[05:48:01] <waqas> I don't have much idea of how good any of those are
[05:49:50] <jb> What would be the number of concurrent users at which point you should really be running an xmpp server on a dedicated machine?
[05:51:21] <waqas> Erm, that depends on how much resources are available to the server. CPU usage is usually minimal, so the deciding factor is RAM. As long as RAM is available, it doesn't matter if the machine is dedicated or not.
[05:52:35] <jb> So the xmpp server basically does matchmaking and presence, whereas the message traffic is peer to peer?
[05:53:26] <waqas> No, everything is via the server, including messages. File transfers and audio/video can be either direct or proxied by the server.
[05:55:50] <jb> So text message traffic is client-server-client whereas binary and a/v traffic is client-client?
[05:57:07] <waqas> That's roughly correct.
[05:57:46] <jb> Going back to server resources, how much RAM would you say is needed to serve 10,000 concurrent users?
[06:00:29] <waqas> Depends on server configuration. TobiasFar did some benchmarks here for Prosody: http://ayena.de/files/prosody_memory-0.8rc1.png
[06:01:24] <waqas> That benchmark is simplistic, with empty user accounts. Actual usage would be higher.
[06:03:31] <waqas> That image has 100 users/MB with encryption enabled, compression disabled. That red line which is higher is with compression enabled.
[06:03:49] <waqas> Err, sorry, 10 users/MB
[06:07:46] <jb> Looks like 100MB for 1,000 users without compression ... or 10 users per MB as you say.
[06:08:38] <jb> How much more RAM per user would you say, in a real-world scenario?
[06:12:10] <jb> Are you there?
[06:14:25] <waqas> I just checked the stats for a server (thiessen.im), and it seems to be sitting at 250MB with around 80 c2s and 160 x 2 s2s connections.
[06:15:44] <waqas> c2s = client-2-server, s2s = server-to-server. s2s connections require two network connections each.
[06:18:10] <jb> That's roughly 3MB per user ... would you say that's about right?
[06:18:51] <waqas> In this case, there are the s2s connections too, which are probably all encrypted, and many compressed.
[06:20:04] <jb> Does an xmpp server make 2 sockets for every user, regardless of whether multiple users are on the same xmpp "domain"?
[06:20:50] * teo1 left the chat.
[06:20:53] * teo1 joined the chat.
[06:25:18] * waqas left the chat.
[06:26:43] <jb> Anybody there?
[06:33:02] * waqas joined the chat.
[06:33:38] <waqas> No, one socket per user.
[06:33:58] <waqas> Per jabber supports federation, which allows a user on one server to talk to a user on another server.
[06:34:47] <waqas> In federation, two servers connect to each other, to route these messages. Those connections require two sockets per pair of connected servers.
[06:35:36] <waqas> e.g., I have 100+ contacts in my contact list, most of which are on their own servers.
[06:38:08] <jb> So that why thiessen.im currently has 80 c2s and 160 s2s ... the 160 s2s are the sockets for the federated server connections needed to support the 80 concurrent users?
[06:40:28] <waqas> Yes. Depending on whether your users connect to other servers mostly, or have contacts all on the local server, your numbers would vary.
[06:41:59] <waqas> Most of the people in this room (including me) are connected from different servers by the way.
[06:43:26] <jb> I'd like to build a chat that's very easy for new users to join and use ... also, the client should be a web client, again for ease of use ... do you know any clients that meet those reqs?
[06:44:44] <jb> Another req might be easy integration of a/v chat, also on a web client ... is that possible?
[06:47:41] <waqas> There are a number of web based clients
[06:47:56] <waqas> http://xmpp.org/xmpp-software/clients/
[06:48:26] <waqas> Click on the "Platforms(s)" column title to sort them by platform. The web "Browser" ones are web based.
[06:49:01] <waqas> I don't think any of them do a/v.
[06:50:47] <jb> How mature and how robust is a/v chat on xmpp?
[06:52:01] <jb> Sounds like a/v chat needs a client running on the host's native h/w, or at least a Java client, would you say that's about right?
[06:54:28] <waqas> Hard to say, as I don't use it much personally. I do know that multiple clients have support, and can talk to each other just fine. As for maturity, the support is rather new.
[06:55:46] <waqas> a/v chat in a web based client.. you can use flash to do audio/video capture, and route it through the server. It wont be perfect however, as web browsers are not too amazingly capable of multimedia at the moment.
[06:56:14] <waqas> The gtalk/gmail folks wrote their own browser plugin to support video in gmail chat.
[06:59:50] <jb> Flash player 10 has a/v chat capacity built-in but it's based on something adobe calls RTMFP and not XMPP
[07:00:26] * Alex joined the chat.
[07:03:26] <jb> There are a couple of flash xmpp libs ... xiff3 is one of them ... don't know if anybody's used them to make a good flash client ... do you know?
[07:03:56] <waqas> There were some flash xmpp clients, but I can't remember the name of any
[07:07:22] <jb> How difficult is it to install and operate an xmpp server, such as prosody for example?
[07:08:23] <Kev> Small scale or large scale?
[07:08:51] <Kev> Small scale it's trivial, large scale (not talking about Prosody in particular) you need to be a bit more forward-planning.
[07:09:25] <waqas> And by large scale I think Kev means a distributed/clustered setup.
[07:09:47] <Kev> Right, either many users, or required resilience.
[07:10:27] <waqas> Kev: Does Swift do audio/video by the way? Planned?
[07:10:57] <Kev> It doesn't at the moment. It's a desired feature, but the only thing that's actually *planned* is getting 1.0 out the door.
[07:12:21] <jb> Small or large scale ... let's say we start small, but also want to scale without any major difficulties
[07:13:11] <Kev> Scale to what level?
[07:14:19] <jb> Can we leave that open-ended? ... and assume the thing goes viral ... how do we scale it without major headaches?
[07:15:06] <Kev> Well, assuming you want to go above jabber.org levels (350k registered users, 16k concurrent) or so, you probably need to consider a server with good clustering support.
[07:15:48] <jb> Ok, can you recommend such a server?
[07:15:53] <Kev> Anything less than that and you're probably ok with a server that doesn't cluster.
[07:16:02] <Kev> (And of those Prosody is the best, I suspect)
[07:16:21] <waqas> I'll note that Prosody doesn't do clustering yet (planned feature)
[07:16:56] <Kev> jb: Well, I work for Isode, and I think their server is the best for large deployments, so ... M-Link.
[07:17:04] <Kev> It's non-Free.
[07:17:36] <Kev> (Note that I thought M-Link was the best for a significant length of time before I started working there, so while I'm not entirely unbiased, it's not just corporate loyalty)
[07:18:53] <jb> So which xmpp servers currently support clustering, (other than M-Link) ... and are they currently deployed in large clusters?
[07:20:33] <Kev> Jabber XCP does, I think, and I think ejabberd.
[07:21:30] <jb> Do we know which xmpp server is being used by Google Talk?
[07:21:37] <Kev> They rolled their own.
[07:22:48] <jb> Has anybody used the Google Talk a/v chat? ... do we know whether Google is having any major issues with a/v chat?
[07:23:11] <Kev> I've never used the video, I've used the audio. It works fine afaik.
[07:27:08] <jb> Do we know the technology behind the Google Talk client ... I mean other than xmpp ... for example, flash, or thick server driving thin ajax/html client, etc.?
[07:27:24] <Kev> Which client? They have several.
[07:27:33] <waqas> They have both a flash based and a js based client, and they have a desktop client.
[07:27:50] <jb> Oh ...
[07:27:53] <waqas> And the audio/video support is a browser plugin.
[07:28:14] <Kev> Browser clients typically use BOSH, and just do the XMPP themselves except wrapped in HTML.
[07:28:31] <waqas> s/HTML/HTTP/
[07:29:19] <Kev> Quite right.
[07:29:56] <Kev> Never anwser questions while eating breakfast and playing games :)
[07:30:18] <jb> I wonder why they chose to support so many different clients?
[07:30:39] <Kev> Because they wanted maximum penetration, one would assume.
[07:30:50] <waqas> Kev: Ha, I had just finished breakfast and was about to start a game ^^
[07:33:01] <jb> But from a user's perspective, wouldn't the plethora of different client choices create confusion?
[07:33:21] <Kev> No idea. I'm not confused :)
[07:33:29] <Kev> Well, I probably am confused, but not by that.
[07:34:49] <jb> Right, but none of us are the average Google Talk user demographic, from a technical know-how point of view that is.
[07:35:20] <Kev> I imagine most people use the gmail-embedded client, and don't know there's a desktop one, or that they're using Google Talk.
[07:36:45] <jb> I should go and use Google Talk.
[07:37:50] <jb> So what's the price structure of M-Link?
[07:43:51] <Kev> No idea, I'm an engineer.
[07:44:07] <jb> Right
[07:44:17] <Kev> Happy to poke you to the business dev manager though.
[07:44:33] <jb> Ok ...
[07:45:01] <Kev> brb
[07:46:55] <jb> Were does Cisco fit into the xmpp server landscape?
[07:47:29] <Kev> jb: They have the Jabber XCP server. As far as I know they're only currently providing very large customers like government.
[07:48:10] <Kev> I could easily be wrong about that, though.
[07:49:00] <Kev> Would you like me to put you in touch with Mark? (Bus. dev.)
[07:49:27] <Kev> If so, if you can pass me an email address for you, I'll send you both a mail.
[07:50:21] <jb> Let's say we deployed prosody, and after a while we grow to the point where we need clustering ... how painful is the migration from prosody to m-link?
[07:50:50] <Kev> I've never migrated from Prosody to M-Link, but typically migration from one server to another is significant, but do-able.
[07:51:04] <Kev> If you want complete data coverage, very significant.
[07:51:56] <Kev> e.g. Migrating my home server (say a dozen users) was a matter of writing a python script, running it, running an importer for M-Link, and bringing the old server down and M-Link up.
[07:51:57] <jb> Can you elaborate on complete data coverage?
[07:52:00] * mazzachre joined the chat.
[07:52:08] <Kev> Whereas on jabber.org, it was the work of weeks.
[07:52:15] <mazzachre> Morning.
[07:52:19] <Kev> Morning mazzachre.
[07:52:38] <mazzachre> Is the server connection to tigase still down?
[07:52:40] <Kev> jb: You've got users, their rosters, their private data, their PEP nodes, MUCs, etc.
[07:52:53] <Kev> mazzachre: The problem isn't that it's down - the problem is that it's up :)
[07:53:27] <Kev> We've still got a session open to the server, but it doesn't look like it's listening.
[07:55:45] <Kev> mazzachre: I've put it on today's todo to see if I can sort it out.
[08:00:03] <jb> So jabber.org was originally deployed on prosody, then migrated to m-link?
[08:00:25] <Kev> No, originally jabberd1, then ejabberd, then M-Link.
[08:01:53] * Zash joined the chat.
[08:02:55] <jb> Earlier in the chat it appeared that 350k registered users, 16k concurrent was the inflection point for the transition from a non-clustered to a clustered xmpp server ... what that the reason for the move to m-link?
[08:03:23] <Kev> We hoped it would be more stable, and have better support.
[08:03:53] <Kev> (and it has been)
[08:05:45] <jb> Sounds like stability was an issue at the scale on ejabberd ... can you elaborate on the nature of the stability concerns?
[08:06:24] <Kev> Jabber.org used to crash every couple of days.
[08:06:35] <jb> Oh ... not so good
[08:07:02] <Kev> I note that jabber.org is the world's most hostile environment to deploy an XMPP server, because it's what everyone tests their half-baked clients and bots against, so this isn't saying that a normal server would be so bad.
[08:07:39] <jb> How's it now on m-link ... any similar issues?
[08:07:57] <Kev> We typically need to restart M-Link every 3 weeks or so.
[08:08:00] <Kev> Well, we don't.
[08:08:14] <Kev> M-Link comes back up after a crash in almost all cases - but there's typically a restart every 3 weeks or so.
[08:08:31] <Kev> We're hoping to deploy it clustered before *too* long, which would give much greater resilience.
[08:09:47] <Kev> (Isode (who I didn't work for back then) recommended we deploy it clustered from the start, as that's the prefered resilience model, but we didn't have the hardware to do so. I hope that'll get solved soonish)
[08:10:58] <jb> Speaking of h/w ... what does one need for a single server m-link deployment ... what about a smallish cluster?
[08:11:15] <Kev> It depends entirely on your requirements.
[08:11:32] <Kev> I run my home server on a VPS that does many things, has a total of about 200meg ram and 4GB disk.
[08:11:46] <Kev> And cluster it home to my DSL connection to a virtual machine I run here.
[08:12:20] <Kev> jabber.org is a machine with (I think) 4 cores, and it has 16gig RAM and plenty of space.
[08:14:47] * tkoski joined the chat.
[08:14:57] * tkoski left the chat.
[08:15:11] <jb> And currently jabber.org supports about 350k registered users, 16k concurrent on that single machine m-link deployment?
[08:16:16] * jonas joined the chat.
[08:16:29] <Kev> Yes. It would support more concurrent users, but how many I don't know. It just happens that 16k is about what we experience.
[08:17:56] <Kev> Requirements are largely based on what the users are doing etc, not the number of them.
[08:17:58] * luca tagliaferri joined the chat.
[08:19:04] <jb> Right ... speaking to that ... how competent is m-link in the mmog arena ... where traffic is likely much higher bandwidth than chat?
[08:19:39] <Kev> Shifting <message/>s around it's very good at.
[08:19:48] <Kev> That's actually the easiest job it does, pretty much.
[08:20:08] <Kev> So just 'high traffic' isn't a problem.
[08:20:32] <Kev> (e.g. presence has a much higher cost than messages, because the server has to then fan it out to everyone in the roster)
[08:22:32] * Jonathon Conte joined the chat.
[08:23:04] <Kev> (In case that wasn't clear (I don't know how much XMPP exposure you have), <message/> doesn't mean a chat message - it's the usual wrapper stanza for shunting traffic around (including, but not only, chat).
[08:23:59] <jb> Right ... was aware of that ... any problems sharing the same server between a chat app and an mmog (or several mmog's)?
[08:24:18] <Kev> None that immediately come to mind.
[08:25:44] <jb> Latency is another matter ... chat is not sensitive to it, but mmog very much is ... any issues there?
[08:25:46] <Kev> Why don't I put you in touch with Mark? He's pretty technical anyway, and he can always put you on a call with (probably) the M-Link team lead and me to talk you through stuff more comfortably.
[08:26:09] <Kev> That's something you'd have to test to be sure.
[08:26:54] <Kev> You *might* want to separate out hardware if that's critical, but you'd need to know what latency you need, and test it in your sort of environment to see what setup's going to do best for you.
[08:27:59] * Jonathon Conte left the chat.
[08:28:16] <jb> Is m-link currently deployed on latency sensitive apps such as mmog's ... if so, can you give some examples?
[08:28:17] * Jonathon Conte joined the chat.
[08:28:59] * Jonathon Conte left the chat.
[08:29:13] <Kev> I don't actually know - I know we're in discussion with some games companies, but I don't know at what stage they are. Mark's your person there.
[08:30:22] * nabatt joined the chat.
[08:34:19] * waqas left the chat.
[08:35:21] * waqas joined the chat.
[08:38:26] * Tobias joined the chat.
[08:55:35] * waqas left the chat.
[09:02:39] * Zash left the chat.
[09:09:20] * Zash joined the chat.
[09:09:21] * Zash left the chat.
[09:18:12] * rtreffer joined the chat.
[10:14:29] * jb left the chat.
[10:35:11] * Tobias left the chat.
[10:38:26] * waqas joined the chat.
[10:42:43] * Tobias joined the chat.
[10:50:26] * scippio left the chat.
[10:58:07] * Tobias left the chat.
[11:02:06] * Zash joined the chat.
[11:06:12] * Tobias joined the chat.
[11:17:32] * Treebilou joined the chat.
[11:25:41] * naw joined the chat.
[11:27:56] * naw left the chat.
[11:31:52] * naw joined the chat.
[11:32:12] * naw left the chat.
[11:34:06] * Zash left the chat.
[11:44:33] * Zash joined the chat.
[11:46:00] * Treebilou left the chat.
[12:14:13] * scippio joined the chat.
[12:22:48] * Tobias left the chat.
[12:45:11] * skip left the chat.
[13:02:18] * nabatt left the chat.
[13:10:48] * Treebilou joined the chat.
[13:54:11] * naw joined the chat.
[14:07:55] * Florob joined the chat.
[14:28:37] * darkrain joined the chat.
[14:29:25] * bartender left the chat.
[14:30:18] * bartender joined the chat.
[14:38:56] * jcea joined the chat.
[14:40:26] * Tobias joined the chat.
[14:40:31] * mlundblad_netbook joined the chat.
[14:44:59] * zanchin left the chat.
[14:52:31] * Florob left the chat.
[14:52:33] * Florob joined the chat.
[14:56:01] * mlundblad_netbook left the chat.
[14:56:30] * Alex left the chat.
[15:08:46] * naw left the chat.
[15:09:58] * naw joined the chat.
[15:23:22] * zanchin joined the chat.
[15:34:00] * teo1 left the chat.
[15:40:27] * scippio left the chat.
[15:40:32] * scippio joined the chat.
[15:43:24] * darkrain_ joined the chat.
[15:44:24] * deryni joined the chat.
[15:54:18] * scippio left the chat.
[15:55:10] * scippio joined the chat.
[16:03:02] * mazzachre left the chat.
[16:05:30] * scippio left the chat.
[16:05:45] * scippio joined the chat.
[16:07:41] * waqas left the chat.
[16:16:38] * hawke joined the chat.
[16:20:53] * Florob left the chat.
[16:20:58] * Florob joined the chat.
[16:27:27] * scippio left the chat.
[16:27:38] * scippio joined the chat.
[16:31:03] * teo1 joined the chat.
[16:43:33] * jonas left the chat.
[16:49:11] * louiz’ left the chat.
[16:49:20] * louiz’ joined the chat.
[16:54:08] * nikita left the chat.
[16:56:25] * niekie joined the chat.
[17:10:45] * evilotto joined the chat.
[17:15:35] * niekie left the chat.
[17:20:15] * nikita joined the chat.
[17:23:46] * tofu joined the chat.
[17:26:23] * scippio left the chat.
[17:29:01] * scippio joined the chat.
[17:32:32] * jcea left the chat.
[17:39:49] * Florob left the chat.
[17:39:56] * Florob joined the chat.
[17:56:30] * luca tagliaferri left the chat.
[18:02:27] * scippio left the chat.
[18:13:04] * Zash left the chat.
[18:14:55] * lastsky joined the chat.
[18:36:36] * Florob left the chat.
[18:36:40] * Florob joined the chat.
[19:11:35] <Tobias> a server could also act as a jingle node, right? or won't that be discovered?
[19:12:47] <Tobias> http://xmpp.org/extensions/xep-0278.html#clientcheckownserver this at least suggest that
[19:14:30] * lastsky left the chat.
[19:34:43] * bear left the chat.
[19:34:43] * bear joined the chat.
[19:52:55] * gigam joined the chat.
[20:01:09] * Florob left the chat.
[20:13:34] <Tobias> what where those XEPs about spam and abuse prevention, have they all been deferred?
[20:15:13] * scippio joined the chat.
[20:16:19] <Tobias> doesn't look nice to recommend to implement a bunch of deferred XEPs, does it? :)
[20:17:37] <Tobias> Kev: or what do you think? :)
[20:31:59] <Kev> I don't see why it doesn't.
[20:32:17] <Kev> If the XEPs have interest in them, they can be resurrected.
[20:33:00] <Kev> I was having thoughts about spam and stuff a couple of days ago.
[20:42:48] <Tobias> might be a nice opportunity to get things tested IRL was GSoC prosody project
[20:42:52] <Tobias> *as
[20:53:31] * justin joined the chat.
[21:00:23] * mlundblad joined the chat.
[21:13:34] * Treebilou left the chat.
[21:43:30] * Florob joined the chat.
[22:13:54] * darkrain left the chat.
[22:14:28] * darkrain joined the chat.
[22:34:54] * MattJ joined the chat.
[22:45:03] * Tobias left the chat.
[22:49:15] * jcea joined the chat.
[22:52:23] * Tobias joined the chat.
[22:58:33] * Tobias left the chat.
[23:11:39] * waqas joined the chat.
[23:35:41] * MattJ left the chat.