Page 1 of 2

It's not lag -- it's completely clogged.

Posted: Fri May 09, 2008 5:15 am
by ichi
The lag has been getting worse recently. Today it completely clogged: minutes were not reset, projects were not processed, and it was nearly impossible to log in for most of the day. On the positive side, animal reproduction/movement appear to proceed as normal. Am I the only one who thinks animal processing is causing the lag/clog? Every place my characters go, they find lots of animals. Even in places where people regularly hunt, there are large numbers of animals. Are all these extra animals clogging the processing?

Perhaps we could reduce animal processing to once a day until the precise cause of the lag/clog is identified and eliminated? Or is it possible to put a low priority on animal processing, so that it proceeds only when there is a low loading on the server?

I would also like to suggest we give a low priority to other nonessential processes (such as decay processing) until the lag/clog has been eliminated. RP is the heart of Cantr, and the lag/clog is making meaningful RP just about impossible. The myriad calculations performed by the server provide a rich environment for us to do the RP, but those calculations are pointless if they kill the RP.

Posted: Fri May 09, 2008 5:57 pm
by Doug R.
ichi,

There is no single cause of this problem. If there was, the lag would be predictable. The game right now is in a worsening traffic jam. It's not just the lag; there are other things that are broken entirely right now.

Because the server's in a traffic jam, every single process that runs makes it worse. I doubt trimming back on processes would do much at all, since processes that currently take minutes are taking hours to run (for example, at the time I posted this, there were four server processes running, two of which, count characters and kill characters, have been running for 13 and 2 hours respectively. Normally they take minutes.

When the game works, it works fine. It had been working fine for several months now. Something caused the traffic jam, and now everything's broken and all crapped up. Obviously, programming doesn't know what caused the jam, and the fact that it doesn't happen regularly suggests it's something that happens fairly infrequently, making it much harder to pin down. If it were a normal game process that happens every day, like animals or rot, the server would be like this 24/7/365, which it is not (when it's working fine).

I've personally given up. I can't play. I can't do my shepherd work. I can't even do what's in my power to help, because a) I don't even know if the animals have reached their game max (the brokenness I've mentioned), and b) I can't cull because the server's so slow a cull would slow the server down even more. I'm tired of bugging the only programmer Cantr has left, and I'm sure he's sick to tears of being bugged. He knows its f-ed up. Jos knows it's f-ed up. I'll just find something else to do until it's not f-ed up.

Posted: Fri May 09, 2008 7:27 pm
by Zanthos
perhaps a temporary halt on outside influences is in order? maybe closing off entry into the game for a day or 2, ceasing project progress and animal movement, halting hunger progression, ect... and just letting everything else run its course, until things are working as normal again?

Posted: Fri May 09, 2008 8:15 pm
by NaruShadow
When the game works, it works fine. It had been working fine for several months now. Something caused the traffic jam, and now everything's broken and all crapped up.


I think you need more server space. Either that or the server has been hacked.[/quote]

Posted: Sat May 10, 2008 11:43 pm
by psymann
Doug R. wrote:When the game works, it works fine. It had been working fine for several months now. Something caused the traffic jam, and now everything's broken and all crapped up


It's sad that your expectations are so low :(

Yes, at the moment, it's unplayably clogged up. But it's not been fine for several months. It's been just-about-ok for several months.

Even when Cantr is running at its top speed, it's still distinctly slow. If a page loads instantly (ie less that one second between me clicking on the link and the page having completely appeared) then I'm surprised. It rarely happens. When Cantr is at its best, it still seems to take a couple of seconds for most pages. And that's been the case pretty much ever since I started playing in Autumn 2006. It's at least playable when it's taking a couple of seconds per page, but it's still abnormally slow compared to other sites. Add that to some frustrating interface design points, and Cantr is easily the most irritating website I use.

It may just be that the choice of programming language is just not a fast one, or it may be that there has been a little bit of lag at all times, but it does frustrate me that I'm put off playing the game just because of it. I wish the staff members much luck in improving things, and hope they're successful :) Can't be an easy job.

psymann

Posted: Sun May 11, 2008 1:16 am
by kasuo
Sometimes when I'm bored and waiting for a page to load, i glance at the source. It looks like there are a lot of redundant html tags that can be a lot better summarized with CSS. After a while, this would add up to a good amount of bandwidth saved.

Posted: Sun May 11, 2008 10:13 am
by Tiamo
Bandwidth is not the problem, kasuo. It is the server's processor that is the bottleneck here. When processes that usually take seconds now take half an hour or more there is a lot of waiting involved, that is processes waiting to continue.

One possible reason could be an extreme amount of swapping when switching between processes (due to lack of internal processor memory), another could be processes that monopolize database access (do not allow other processes to access the database while the process is not finished), even when the process is waiting for some parallel activity to complete (hard disk read/write, network activity, printing, etc.).

The number of database actions needed for generating a single HTML-page seems rather high; more efficiency in database access could be the key to solving the problems.
Unfortunately that would mean a complete revision of all software, too big a task for the current staff i fear.

Posted: Tue May 13, 2008 7:32 pm
by kasuo
Tiamo wrote:The number of database actions needed for generating a single HTML-page seems rather high; more efficiency in database access could be the key to solving the problems.
Unfortunately that would mean a complete revision of all software, too big a task for the current staff i fear.


Has there ever been a proposal for others to take a glance at the database architecture for suggestions on improvement?

Posted: Tue May 13, 2008 8:30 pm
by Doug R.
The hierarchy of the Cantr administration is such that non-staff are not allowed access to anything. To do anything, you need to join a department, in this case, Programming, and then you'd need to be trained. Except the chair (and only active full member) is too busy fixing the game when it breaks, so he doesn't have time to train anyone (and I imagine the amount of training required, unlike most departments, is considerable, especially if the aspirants don't know the language cantr is written in).

It's a catch-22, and the root of 99% of Cantr's problems.

It's notable that while most of the Cantr departments are actually thriving, the two that require some level of expertise, Marketing and Programming, are crippled. This is because they have a much smaller talent pool to draw from, and the obligation to join a department most assuredly turns away people that could help, but view joining a department as a commitment that they aren't prepared to make.

Unless something major happens, this isn't going to change, and when Marol decides it's time to move on from Cantr, the game will be finished.

Sorry to sound doom and gloom, but this has been the reality of the situation since CJ decided to move on. Enjoy Cantr while it lasts.

Posted: Tue May 13, 2008 9:08 pm
by Zanthos
its a shame, because there isnt any other game like cantr. if i had the programming experience i'd consider joining progd, but i dont, nor do i have the time as a fulltime university student.

Posted: Tue May 13, 2008 9:30 pm
by Yosarian
Is there some reason it's set up like that? I realize I'm new here, but it seems like there should be some way for you guys to accept help from voulenteers with computer programing experence who want to help fix the code.

Posted: Tue May 13, 2008 9:31 pm
by Voltenion
Vacation is near... I'll have plenty of time (at least that's what I think). I know nothing about progamming but I heard they teach, or would teach if they had the time... Wharever, I'm going to apply to something this summer. I need to help :) And by the way: does anyone know what I should apply too? What department needs more help?

Posted: Tue May 13, 2008 9:41 pm
by Zanthos
prog D, the game needs to be fixed before marketing department can help it.

Posted: Wed May 14, 2008 12:08 am
by TerribleTadpole
We need to recognise that if the lag has a simple cause or a simple solution then it would have been found and fixed. It's likely caused by conflicting table locks (both updates and inserts cause locks) cause processes to deadlock. While each "tick" process can probably run efficiently by itself, if another starts and overlaps then there will be contention for processor time, memory, disk I/O and data locks. Once that starts you're likely to have a cascade of concurrent processes all piling up and adding to the problem.

When you're pages take a long time to load, it's most unlikely to be because the queries are thrashing around gathering up data. It's far more likely that the queries are held waiting for some locked data to be released.

The solution to this kind of situation frequently involves a vast amount of re-coding. One example would be to change single, long-running UPDATE queries that lock large parts of tables into a series of smaller UPDATE queries, or possibly in to a loop walking a cursor updating each row individually.

Let's just accept that whatever the problem is, Jos wants to find it. And if he's found it then he wants to fix it. And if he's fixing it then he's putting as much into it as he can.

In the meantime I find that the game is only slow sometimes. If it's difficult, I leave it and come back later. That's hard on those of you who don't have the luxury of being able to play at a time that suits the game rather than your life. And I guess Jos is sorry about that. But nagging doesn't help so let's not do it.

Posted: Wed May 14, 2008 12:22 am
by wichita
As has been stated, but is always getting lost in further discussion, we need applicants to the programming department that already know how to code PHP and MySQL in order to really address the staffing needs.


If you already know how to code (not just planning to learn it or in the middle of learning it) we could really use the help.