Validation Debugging

RE: Validation Debugging

So what is happening here is that during the save command both the planet and the ship which you transferred too should be sent to the api. That way the system can check that the data is valid between those two ships. (we never checked this in the past, which is what sometimes led to data being lost or gained)

When you begin a transfer both the ship and the planet are marked as changed.
ship.changed = 1planet.changed = 1

When you close the window the save command is called which then looks for all planets and ships with changed > 0 and then includes them in the call to save. For some reason, that is not happening in this special case.

Only one save command can execute at the same time and when the command starts all the items in that save are marked as .changed = 2 (meaning save in progress)

When a save is successful all the items with .changed = 2 are set back to 0.

Somehow, the ship in this transfer scenario is not being sent to the save command. Are you familiar with firebug for firefox or the inspector tools in Chrome? What I would like to do next time you get this error, is inspect the ship data for the ship that was not sent (the transfer target) to see what the value of .changed is. That would help me know the reason it is not being sent.

2514 days, 17 hours, 25 minutes ago

RE: Validation Debugging

Another thing to check.

When you get the data validation issue. Can you post the previous 2-3 /game/save calls leading up to the error.

2514 days, 2 hours, 44 minutes ago

spiper

RE: Validation Debugging

I ran across something that needs to be validated. The game save allowed me to set a negative tax rate. Bad scripting on my part but the turn actually ran that way. I ended up with a negative megacredit amount after the turn ran. As soon as I clicked to build or clicked to set taxes the numbers reset. It actually built a negative number of buildings.

Stephen

2513 days, 23 hours, 35 minutes ago

RE: Validation Debugging

I received another data validation error while playing a Crystalline surrounded scenario. I did some fairly extensive tracing regarding attempting to move cargo to the Medium Freighter, MDF 9.

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107650&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Ship9=Id=9&Name=MDF%209&Neutronium=19&Duranium=57&Tritanium=34&Molybdenum=109&MegaCredits=0&Supplies=0&Clans=0&Ammo=0&TransferNeutronium=0&TransferDuranium=0&TransferTritanium=0&TransferMolybdenum=0&TransferMegaCredits=0&TransferSupplies=0&TransferClans=0&TransferAmmo=0&TransferTargetId=0&TransferTargetType=0&TargetX=1156&TargetY=2149&FriendlyCode=mpd&Warp=9&Mission=0&Mission1Target=0&Mission2Target=0&Enemy=0&Waypoints=&ReadyStatus=1&keycount=11&_=1349207739796 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

... 16 orders later while mostly stopping further web mine laying ...

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107667&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Ship50=Id=50&Name=Emerald%2050M9&Neutronium=201&Duranium=0&Tritanium=0&Molybdenum=0&MegaCredits=0&Supplies=0&Clans=0&Ammo=255&TransferNeutronium=0&TransferDuranium=0&TransferTritanium=0&TransferMolybdenum=0&TransferMegaCredits=0&TransferSupplies=0&TransferClans=0&TransferAmmo=0&TransferTargetId=0&TransferTargetType=0&TargetX=1095&TargetY=2364&FriendlyCode=mdq&Warp=9&Mission=1&Mission1Target=7&Mission2Target=0&Enemy=0&Waypoints=&ReadyStatus=1&keycount=11&_=1349207867184 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107668&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Ship8=Id=8&Name=Thunder%208&Neutronium=84&Duranium=0&Tritanium=0&Molybdenum=0&MegaCredits=0&Supplies=0&Clans=0&Ammo=29&TransferNeutronium=0&TransferDuranium=0&TransferTritanium=0&TransferMolybdenum=0&TransferMegaCredits=0&TransferSupplies=0&TransferClans=0&TransferAmmo=0&TransferTargetId=0&TransferTargetType=0&TargetX=1159&TargetY=2149&FriendlyCode=ntp&Warp=9&Mission=3&Mission1Target=0&Mission2Target=0&Enemy=0&Waypoints=&ReadyStatus=0&keycount=11&_=1349207871550 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107669&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Planet1=Id=1&FriendlyCode=362&Mines=100&Factories=250&Defense=20&TargetMines=0&TargetFactories=0&TargetDefense=0&BuiltMines=0&BuiltFactories=0&BuiltDefense=0&MegaCredits=2479&Supplies=1940&SuppliesSold=0&Neutronium=3874&Molybdenum=1368&Duranium=1338&Tritanium=940&Clans=30904&ColonistTaxRate=0&NativeTaxRate=0&BuildingStarbase=false&NativeHappyChange=3&ColHappyChange=7&ColChange=0&ReadyStatus=0&Starbase7=Id=7&Fighters=0&Defense=0&BuiltFighters=0&BuiltDefense=0&HullTechLevel=8&EngineTechLevel=1&BeamTechLevel=1&TorpTechLevel=8&HullTechUp=0&EngineTechUp=0&BeamTechUp=0&TorpTechUp=0&Mission=4&ShipMission=0&TargetShipId=0&BuildHullId=0&BuildEngineId=0&BuildBeamId=0&BuildTorpedoId=0&BuildBeamCount=0&BuildTorpCount=0&IsBuilding=false&ReadyStatus=0&keycount=12&_=1349207875670 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107670&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Planet1=Id=1&FriendlyCode=362&Mines=100&Factories=250&Defense=20&TargetMines=0&TargetFactories=0&TargetDefense=0&BuiltMines=0&BuiltFactories=0&BuiltDefense=0&MegaCredits=2479&Supplies=1940&SuppliesSold=0&Neutronium=3874&Molybdenum=1368&Duranium=1338&Tritanium=940&Clans=30904&ColonistTaxRate=27&NativeTaxRate=0&BuildingStarbase=false&NativeHappyChange=3&ColHappyChange=-14&ColChange=0&ReadyStatus=1&keycount=11&_=1349207885937 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

... and now for the command that actually fell through.

GET http://api.planets.nu/game/save?jsoncallback=jQuery171037789074309047455_1349207107671&gameid=31208&playerid=1&turn=7&version=1.31&savekey=86933e43-e8f6-4f83-bfc1-3ca3683c7d86&saveindex=2&Planet302=Id=302&FriendlyCode=751&Mines=17&Factories=43&Defense=0&TargetMines=0&TargetFactories=0&TargetDefense=0&BuiltMines=0&BuiltFactories=0&BuiltDefense=0&MegaCredits=0&Supplies=87&SuppliesSold=0&Neutronium=72&Molybdenum=27&Duranium=0&Tritanium=0&Clans=513&ColonistTaxRate=0&NativeTaxRate=0&BuildingStarbase=false&NativeHappyChange=2&ColHappyChange=7&ColChange=0&ReadyStatus=1&keycount=11&_=1349207901901 HTTP/1.1
Host: api.planets.nu
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://play.planets.nu/
Cookie: __utma=56797419.271218994.1349200634.1349200634.1349203028.2; __utmc=56797419; __utmz=56797419.1349200634.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

jQuery171037789074309047455_1349207107671({"success":false,"error":"Data validation error. Neu value does not match - 86 to 72. Dur value does not match - 57 to 0. Tri value does not match - 34 to 0. Mol value does not match - 136 to 27. "})

Responses are not listed except for the very last command that fell through. I also did not notice any errors in FireBug, either.

2513 days, 23 hours, 19 minutes ago

RE: Validation Debugging

Which ship did you transfer from/to when the error occurred? Was that ship saved in a previous command?

2513 days, 22 hours, 46 minutes ago

RE: Validation Debugging

Ship MDF 9 is loading minerals from the planet Alcatraz #302, in game #31208, at the time of the error.

2513 days, 22 hours, 7 minutes ago

RE: Validation Debugging

Ok, so basically you are saying that ship 9 was saved and then 16 other commands when through and then planet 302 was saved. When you came to visit it later?

2513 days, 21 hours, 36 minutes ago

RE: Validation Debugging

Yes. It seems that those 19-20 intermediate orders fell through. Ships that I tried to order to stop laying web mines, for example, did not stop laying until I ordered them a second time around.

2513 days, 21 hours, 30 minutes ago

RE: Validation Debugging

But all those orders recieved a "success" answer?

2513 days, 21 hours, 21 minutes ago

RE: Validation Debugging

Yes they all did.

2504 days, 2 hours, 46 minutes ago

RE: Validation Debugging

Chapter 2

Yesterday, I played a full session through Moriak Sector 0017. The most annoying problem is that I get a delayed "order corruption". After opening the turn just now, some of my orders were silently dropped, and others were only partially dropped. This especially becomes an egregious problem when fighting a crystalline player, or when trying to lay minefields to mitigate cloaking ships.

As part of the partial order drops, planet Charmed World #485 lost 280 clans due to the partial order drop in which it was trying to transfer to the ship LDF 261. The ironic thing is that only the clans were lost. The rest of the cargo I tried to transfer (minerals and money) was simply put back on the planet.

As part of previously narrowing things down, it appears that the best workaround is to exit and reopen the turn after each and every ship-to-planet transfer.

@joshua: Fortunately, I captured and saved the full session using Fiddler2. If you want, I can email you the SAZ archive.

2504 days, 0 hours, 33 minutes ago

RE: Validation Debugging

I am trying to figure out why you get these errors so often, when other (like me) can play through dozens of turns without ever seeing anything like this. Is your connection a little shaky? What browser/OS are you using? I want to duplicate your situation so I can make this happen to me.

2503 days, 23 hours, 7 minutes ago

RE: Validation Debugging

I know it is extremely annoying to reproduce. On another machine, I never was able to reproduce these problems, either, despite using the same OS (Windows XP) and browser (Firefox 15). Last time I checked, I get these problems even with no add-ons. Heck, even the top two Taurus War candidates were reportedly screwed over.

At one point, I used to play the game in Chrome, but I (mostly) stopped playing because add-ons (namely Ad-Block Plus) were breaking the zoom ability, and I did not wish to disable the Ad-Blocker.

Once in a while the connection quality drops, but usually this is limited to a few other servers at a time. After all, the connection quality is only as good as the weakest link. Only occasionally do I actually see a "disconnected" tab while playing, which IMHO is a bit too subtle.

The closet game I would want to touch (barring any live games) to do this test with is the Crystalline surrounded game #31208, since it is the only single-player game I'm currently maintaining that is already affected by the "double marked ready" bug.

The "double marked ready" bug is unpredictable in that it sometimes does not occur, but reproducible in that when it does occur, it always does so as a result of clicking the Ready button, causing the player to sometimes receive credit for hitting the Ready button twice (or more). Related to it is that when you make a new single-player game on the campaign platform, sometimes an extra copy (or possibly two) is made with the same settings.

2502 days, 3 hours, 9 minutes ago

RE: Validation Debugging

Based solely on the testing I've done myself, it appears that I pinpointed the problem, so far, to Trend Micro Titanium. I suspect that it is filtering some of these packets as I do my turn, causing some of the orders to be dropped. Unfortunately, I didn't realize this at the time, so I excluded Trend Micro from my SAZ archive.

Last night, I played a full session of Nekkar on my notebook mini with Windows 7, which has the light-weight Avira in order to accommodate its 1GB RAM. No dropped orders, despite playing on a wireless network.
The night before, I played a full session of Doggle on a desktop machine with Trisquel GNU/Linux. No dropped orders.
Three nights before, I played a full session of Moriak on the very same machine, but this time on Windows XP with Trend Micro. Some orders did get dropped, and a 280-clan transfer was also lost, preventing me from developing another planet.
Likewise, my brother has Trend Micro on Windows XP, so when he does his turns, some of his orders get dropped, too.
My father's experience is dropped orders on his machine with Trend Micro, no dropped orders on his machine with AVG. Windows XP in both cases.

-Paul

2501 days, 18 hours, 32 minutes ago

RE: Validation Debugging

Wow, that might provide some insight, I'm going to install trend micro and see if I can reproduce it.

2500 days, 19 hours, 50 minutes ago

dungeonmaster

RE: Validation Debugging

Ubuntu 10.04 LTS Lucid Lynx.
Firefox 16.0.1 plain security settings.
Last hostrun I had more bugs, once again commands not being processed, these are not mistakes, they're simply not processed commands. It's going to cause me a lot of pain in the championship match sometime, I can just feel it.

2498 days, 22 hours, 50 minutes ago

RE: Validation Debugging

I decided to do another turn on Moriak using XP Trend Micro (with reduced security settings). I even tried adding Planets Nu to the whitelist (using the address pattern http://*planets.nu/*) Not as bad as last session, but a few orders still got dropped.

I've modified the pattern slightly (*.planets instead of *planets) and reverted the rest of the security settings to see what happens the next time I get another turn to play.

2498 days, 22 hours, 43 minutes ago

RE: Validation Debugging

This is definitely an interesting analysis. I wonder if it is due to the very long requests (long query strings with all the save information) we use. We had to override all the security settings on our api servers to make it work like that, but it allows us to do cross domain validation without any problems. It's a fairly new web technique.

2497 days, 18 hours, 58 minutes ago

RE: Validation Debugging