TOADWATER IS DEAD! LONG LIVE TOADWATER!
Toadwater Welcome, Guest. Please login or register.
Did you miss your activation email?
October 20, 2019, 04:20:24 pm
Home Login Register
News: Jinjur finished fertilizing a field for Mopsi

Toadwater  |  Toad Server Cage  |  General Support  |  Topic: Map File Fun 0 Members and 1 Guest are viewing this topic.
Pages: 1 [2] 3 Print
Author Topic: Map File Fun  (Read 4954 times)
Zim Foamy Fan
Centivelian
----------
*

Karma: 275
Offline Offline

Posts: 4368



WWW
« Reply #17 on: April 23, 2008, 04:09:42 pm »

No, that would be horrible and probably the most destructive thing you could do to the game.
What he said.
Logged

macroing is cool

[00:30:00] <+EmperorWu> don't finish
[00:30:02] <+EmperorWu> I'm in.

[00:24:20] <Travis> I wasn't smart enough to back it up.
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #18 on: April 23, 2008, 04:46:00 pm »

Okay, this is essentially a revision of the information from my 1st post, reflecting my knowlege as it stands now:

We have a 4 byte text header (TWC6) followed by an unsigned long telling us how many bytes the first section's data is (useful to improve fficiency without buffering the entire file)

Next We have a list of filenames that can be tokenized using ^ (5E), null terminated

Then, for each WS we have data on, we have have a six byte header telling us which WS it is, then an unsigned short telling us how many tiles we have stored for this particular WS

For each of those tiles, we have 14 bytes of data, arranged as follows:


[0]     co-ords x (byte)
[1]     co-ords y (byte)
[2][3] image idx 1 (unsigned short)
[4][5] image idx 2 (unsigned short)
[6][7] image idx 3 (unsigned short)
[8][9] scale flag, for image 2 (unsigned short) **if this isn't an unsigned short, [9] may be otherwise useful
[10][11] unknown
[12][13] unknown

After analysing a map file with approx 10,000 tiles saved, poking it with a hex editor and loading it into my client (without corrupting it once!) several times, it appears that the three slots for images are utilised as follows:


image 3: not seen used yet - not even for NPCs
image 2: trees, fences etc that sit on top of things (resizable by byte 8)
image 1: base tiles, sand, snow, CRF etc

On simple squares: bare sand or CRF for example, both images are the same rather than the 2nd being set to 00 00 (likely because in all files I've seen, the image at index 0 is a weather image)

Trees with idols (and presumeably other adornments) on are stored as follows:
[2][3] index of Tree image
[4][5] index of idol iamge
[6][7] 00 00

NPCs are saved as follows:
[2][3] index of ground image (sand/snow etc)
[4][5] index of NPC.gif
[6][7] 00 00

It looks like the chopped tree resize bug/feature might be because only the 2nd image can be resized, which on tap/idol/bh trees is the adornment, the tree being image 1. Also, its convenient that we can't chop trees with adornments ;)

Matt:

I still have no idea what the last four bytes do... and could use confirmation that byte [9] (10th byte) is the 2nd half of the scale flag. If you get the chance to find some answers from the code doing the writing, I'd be very grateful.

I plan on tidying up the test parser over the next couple of days (free time is tight) and uploading its code for anyone who has use for it, and then moving on to displaying it (either in its entirety or by region). Once thats done, I'll have a look at generating big, stitched together, map images that people can photoshop when posting base images etc.
Logged
Twisti
Guest
« Reply #19 on: April 23, 2008, 05:06:12 pm »

You should make something that converts the map file to XML and back, that way, people could write fancy stuff without having to parse the map themselves.
Logged
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #20 on: April 23, 2008, 05:41:21 pm »

Thats certainly feasible, an appropriate XSLT may even make maps renderable in browsers.
Logged
Bryan
Centivelian
----------
*

Karma: -180
Offline Offline

Posts: 924



« Reply #21 on: April 24, 2008, 04:04:26 am »

I would love to see this happen.
Logged

[23:29] <[rebel]Epik> http://www.toadwater.com/index.php?action=profile;u=22454
[23:30] <[rebel]Epik> We have competition
[23:31] <@Jessy> thats not competition thats lunch.
Matt Siegman
über programmer
Apprentice
----------
********

Karma: 12
Offline Offline

Posts: 5024


C++ ftw!


WWW
« Reply #22 on: April 24, 2008, 05:13:53 am »

The code is on my other computer. The next time I turn it on I'll take a look at the map file serializer. If I remember correctly, it's a sucky piece of code that is very confusing.
Logged

Matt Siegman
über programmer
Apprentice
----------
********

Karma: 12
Offline Offline

Posts: 5024


C++ ftw!


WWW
« Reply #23 on: April 25, 2008, 08:00:45 am »

Percent is a word. (like the others, word = 2 bytes)
The final dword is the time. (dword = double word)
There are indeed three layers. They are used, you just don't know where Wink

PS: there are six versions of this history file before the one we're at now,.
Logged

spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #24 on: April 25, 2008, 12:36:17 pm »

Thanks, Matt. This is what I needed.

The client seems to render the third layer on top of everything else, (I made a very scary giant idol fence) so I'll mimic that behaviour. Hopefully, wherever its used will still look sensible.
Logged
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #25 on: April 27, 2008, 09:27:49 pm »

Okay, I'm now parsing the files reliably, organising the tile data into a coherent structure, (although have only partially sorted out the retrieval) and have just started to display map data.

The image below is showing the display of only the *background* image, - haven't got round to superimposing the other images



The next one shows us what things look like shrunk down: - as you can see I haven't looked at resizing the images properly, leaving them cropped (which looks strangely nice, but wont work for trees etc I dont think)



Hopefully, I'll make some more progress soon Smiley

Logged
Matt Siegman
über programmer
Apprentice
----------
********

Karma: 12
Offline Offline

Posts: 5024


C++ ftw!


WWW
« Reply #26 on: April 27, 2008, 10:04:16 pm »

How are you storing the map files in memory? One of the big problems that a lot of peoples have run into when trying to build some sort of map viewer is memory usage, which doesn't seem like a big deal until you realize that you're dealing with potentially millions of tiles.

Also, what language are you writing this viewer in?
Logged

Bryan
Centivelian
----------
*

Karma: -180
Offline Offline

Posts: 924



« Reply #27 on: April 27, 2008, 10:17:08 pm »

I'm guessing Java Matt..
Logged

[23:29] <[rebel]Epik> http://www.toadwater.com/index.php?action=profile;u=22454
[23:30] <[rebel]Epik> We have competition
[23:31] <@Jessy> thats not competition thats lunch.
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #28 on: April 27, 2008, 10:21:16 pm »

Im a java programmer, so things like efficiency and memory usage usually get ignored.

Tiles are being stored as objects (or for you, structs) containing 4 strings (image file names) and co-ordinates, an int (scale) and a long (epoch timestamp). In theory, that should be around 40 bytes per tile, on average,

with 10,000 tiles (size of my map file), Its using approx 19 Meg of memory, and 20 Meg of VM, which isnt all that far off the minimum amount java takes up, so I expect it'll scale tolerably

lets say tiles take 50 bytes to make things simpler and allow for overhead. That works out at arond 21,000 tiles per megabyte of memory. Rounding that down to 20k, would make a million tiles take about 50 meg, or 70M taking into account the JVM's overhead.
Logged
Twisti
Guest
« Reply #29 on: April 27, 2008, 11:16:35 pm »

I actually wrote an image cache just for TW, that only loads every image once. I'll post it here tomorrow if you want. That way, you can ditch the string, and the costly conversion, and just put in some Image background; which is only 4 bytes total, and much more efficient.
Logged
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #30 on: April 28, 2008, 12:43:29 pm »

I was wondering whether to go back to merely holding the image index, which should half the memory required per tile, without making me too much work.

Edit:

Actually, it appears that java is only holding a reference to a shared String object in each Tile, making any change unneccessary.
« Last Edit: April 28, 2008, 04:47:02 pm by spheric » Logged
Dragoon
Centivelian
Centivelian
----------
*

Karma: 167
Offline Offline

Posts: 16249


WWW
« Reply #31 on: April 30, 2008, 03:04:43 am »

Why don't you just get the code from Siegman and find out what's locking it down to those two continents and just open it up?
Logged
spheric
Centivelian
----------
*

Karma: 14
Offline Offline

Posts: 339



« Reply #32 on: April 30, 2008, 11:22:46 am »

Because I suck at C/C++ Wink
Logged
Matt Siegman
über programmer
Apprentice
----------
********

Karma: 12
Offline Offline

Posts: 5024


C++ ftw!


WWW
« Reply #33 on: April 30, 2008, 03:21:41 pm »

Whose code is locking what down to two continents?
Logged

Pages: 1 [2] 3 Print 
Toadwater  |  Toad Server Cage  |  General Support  |  Topic: Map File Fun
Jump to:  


Login with username, password and session length

The Toadwater Forums and Game are Copyright © 2014 The TW Development Group
Powered by SMF 1.1.1 | SMF © 2006, Simple Machines LLC