Ladder system update. This is a followon to the Ladder...

« Back

1220 days, 10 hours, 57 minutes ago

Ladder system update. This is a followon to the Ladder...

Ladder system update.

This is a followon to the Ladder system thread that was started by Azzazzello about a year ago, and "lost" when the feed got messed up back in December.

http://planets.nu/discussion/i-am-starting-a-campaign-to-lobby-the-gods-of-this-site-to

I made a few updates to the documentation of the proposed system. Specifically, I have updated the score aging for idle players to match the 3%/month used by Planets.nu. Additionally, I added a proposal for a penalty for pre-game alliances.

http://planets.nu/#/howtoplay/Ladder

I've also been informed that a significant amount of preliminary data will be available for testing soon. This will allow the calculation code to start being written.

1217 days, 5 hours, 7 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

Sorry for the long wait, everyone.

As @Whisperer said, the data extraction is nearly complete. I've sent Whisperer a sample of about 500 games, and will probably be ready to send the rest (roughly 2300) this weekend.

If anyone else wants to try their hand at number crunching, just send me your email via PM.

1216 days, 16 hours, 31 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

As with any complex project, we're finding and fixing problems in the data extraction process. Overall, it's better than I normally see from a first pass. Thanks @Ericlavigne.

Now I need to brush up on my YACC/LEX so that I can parse the data for use in "C" code.

1216 days, 15 hours, 53 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

Ouch. Can I talk you into using anything other than C? Even C++ would let you parse an EDN file with a few lines of code, and you could still pretend you're in C for the rest of the program.

https://github.com/shaunxcode/edn-cpp

Alternatively, I could convert the files to JSON and you can try this library. Like anything in C, it looks a bit painful, but still a lot less trouble than writing your own YACC parser.

http://zserge.com/jsmn.html

There are definitely some problems remaining on the data extraction side, but nothing that should prevent progress on the calculation code. Current extraction code recognizes only one player as winner in a team game (probably an easy fix). Current extraction code also skips 20% of games due to malformed JSON (that one may be more difficult). Good odds that these problems are fixed within a week.

I'm looking forward to seeing how these "ladder" rankings compare with the official rankings.

1216 days, 15 hours, 20 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

Good job! I wouldn't focus on extraction, when the Site Admin take over / step in, they are in a better position to handle data extraction issues :)

We should move forward and try to publish the results of the current rating algorithm and discuss this output :)

1216 days, 15 hours, 12 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

> Can I talk you into using anything other than C?
> https://github.com/shaunxcode/edn-cpp

Thanks. I'll look at that, but no promises.

> a lot less trouble than writing your own YACC parser.

I've done it a few times previously, and I have another project in my queue that will need YACC/LEX parsing for the command/control language. It's not that bad.

Please stick with the EDN you're working with.

> nothing that should prevent progress on the calculation code

Absolutely correct.

> I'm looking forward to seeing how these "ladder" rankings compare with the official rankings.

As am I. I expect them to be closer to the current standings than to the old standings, but there will still be some significant differences.

We'll know in a month or so.

1216 days, 15 hours, 1 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

@Frostriese,

> when the Site Admin take over / step in, they are in a better position to handle data extraction issues

In all honesty, I don't expect the site admins will want to take this over for a few years. They recently juggled the ranks, and probably don't want to do it again any time soon. Also, they would want to verify that the system is stable over time before they pull it in.

> We should move forward and try to publish the results of the current rating algorithm and discuss this output

That's in the plans. There are a few undecided items in the list (http://planets.nu/#/howtoplay/Ladder). Once we have code and data, we can use the results to determine which settings give the closest match to recent game results.

Of course, we all expect Emork to be at the top of the overall list, so anything that puts him elsewhere is immediately suspect. I believe the Mercenaries list is also a good (not perfect) reference point for overall ranking.

If the overall ranking is close to the above, I'll consider it ready for preliminary publishing. It should also give valid race rankings.

1216 days, 14 hours, 34 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

> I believe the Mercenaries list is also a good (not perfect) reference point for overall ranking.

I just took a quick look through the Mercenaries Leaderboard, and I believe that there are a few issues with the order of players who have the same average-rank score. Examples of this are positions 14 & 15, and positions 21 & 22 (there are others).

Because of that sort of issue, not only do I not expect a perfect match to the Mercenaries Leaderboard, but I'd see a perfect match as an indication that the settings need some minor adjustment.

1215 days, 16 hours, 27 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

In case anyone is wondering, @Mule is being kept up to date on the progress of the Ladder system.

817 days, 14 hours, 0 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

It's time to bring this thread back to life.

Please feel free to post any comments or suggestions about the current design (http://planets.nu/#/howtoplay/Ladder).

CURRENT STATUS:

Short version - we should be about 4-6 months away from having this, as long as I don't get sidetracked.

There was a significant delay because Ericlavigne left. The result is that I have now downloaded most of the games, and will be downloading the rest by the end of the week.

I'm verifying the individual turn files as they come in. Unfortunately I'm finding that a significant number of the turn files are not valid. All of the files where I've found the error (I've only checked a few) have turned out to have simple syntax errors, and should be easy to correct (I'll keep Big Beefer updated). I'll also be checking alternate locations at Planets.nu to see if the turn files are in better shape there. Overall, I expect the validation of data, with correction where possible, to require 2-3 months.

Many of the games are missing turns. In some cases, this won't matter. In others, it could cause the system to not be able to generate a valid score for the game. Any game where the score isn't 100% reliable (according to the parameters in the above link) will not be included. As above, I'll be checking alternate locations for the turn files.

I decided to switch the implementation language from C/YACC/LEX to Python. This will be easier to develop, maintain and port, not to mention much faster to write initially. This is a relatively new language to me, but the learning curve isn't bad.

At this time, we do not have an effective way to make this information available. While I have a Hosting Service that could serve the data, I don't have a Domain/Host that I'd be willing to use.

817 days, 7 hours, 29 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

I'm not dead yet! :-)

I don't have a lot of time to spend on this right now, but should have no trouble running my old code to produce the extract files we discussed before. When last we discussed this, it could handle almost all of the games. Many have minor syntax errors. Some are just truncated halfway through, but I've created workarounds for those.

Even if you're sure that you'd like to handle data extraction on your own, you may find my code useful for ideas of how to address the syntax issues. In the following code, cleanup-json deals with minor syntax issues and parse-json-with-autotermination deals with bigger issues like the whole file being truncated.

https://github.com/ericlavigne/planets-rating/blob/master/src/vgap/turn_file.clj#L54

817 days, 6 hours, 37 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

@Ericlavigne,

I thought you had moved on. It's good to see you're still around. Perhaps we'll meet in a game one day :)

I had to download the games for other reasons, so using the files for this won't be a problem. There were some issues in the data you previously provided, and putting it together myself will give faster turnaround while I figure out exactly what's needed.

The code you did should help me to put the data together. It should also help me to solve the JSON problems. Most importantly, it should speed everything up.

Thanks.

811 days, 11 hours, 41 minutes ago

RE: Ladder system update. This is a followon to the Ladder...

Just to give the readers an idea as to the scope of this task, I downloaded a total of 3834 game archives (about 200GB) from Planets.nu, using the "/game/loadall" API, over several weeks (I didn't want to swamp the server). After the initial processing, this resulted in:

A. 1 bad game archive (to be investigated)

B. An unknown number of missing turn files (to be investigated)

C. 4,152,441 good turn files

D. 131,962 bad turn files (turn file syntax is invalid)

D.1. 48,153 bad turn-0 files (Big Beefer has informed me that these are useless)

D.2. 83,809 other bad turn files (to be investigated - the code Ericlavigne provided will help)

As games are identified as "known good" (no missing or bad turns), the needed data will be extracted. Only when all games have been verified as good/valid/useful or bad/invalid/useless will the information from the "good" games be combined into the Ladder.

If the turn files were saved, they'd take up about 4TB of disk space (I don't have that much available space right now). Running one pass through all the turn files, including extracting them from the game files, takes at least a week. Combining the game results should take about a day.

Please be patient with the slow progress. This isn't my primary task, and there's a LOT of data to dig through to come up with the results.