Dev Blog
Hi, this is the Stranded III development blog (see also Forum Thread, Comment Thread).
Entry 92 - Map Editor & Map File Format - May 1, 2020
Short Distraction
Corona is all over the place unfortunately. I was fascinated by the numbers and at the same time frustrated that most pages did not show the information I wanted to see.
Therefore I spent some time with working on my own website related to that topic:
CoronaCurve.de
It was a fun experience! I got to know Chart.js and the CSS framwork Bulma in this little experiment - which might be useful in future.
Most other info pages improved a lot by now and provide more detailed information than I do on my little page.
I can't (and don't want to) keep up with that so I promise that I will focus more on Stranded III again
In other words: There are no plans to put more time into CoronaCurve.de!
Map Editor
Back to the actual topic of this blog: Stranded III! There's some progress with the map editor!
You can now transform the terrain with tools. This includes changing height and textures.
When changing height you can (un)tick a little checkbox which controls if the terrain colors should be automatically adjusted based on the terrain height and steepness.
I recorded a little video to demonstrate what that looks like.
Watch the map editor preview video on YouTube!
Please note:
The editor is unfinished and so are the assets. Some models and textures will be changed in future.
There are no LOD models yet to be displayed for far away objects. That's why stuff just disappears in this video.
There are supposed to be preview images in the object list. Those are currently broken / not properly configured.
The items are not properly configured in this video (ugly shiny material properties and weird colliders for physics)!
Terrain normals are not calculated yet, making the terrain look very "unshaded" and flat
Map Format: Storing Heights (warning: technical stuff ahead!)
I also worked on improving the map format. It's a binary format with a lot of little tweaks which help to minimize the file size of maps (and save games as well).
In these days CPUs are really fast. Therefore the bottleneck when working with files is almost always the physical storage (HDD or SSD). It's still a very slow operation to read/write data (in comparison to other tasks that happen on the CPU/GPU directly).
SSDs are much faster than HDDs for sure but there's that issue with limited write cycles. Therefore I want the map files and save games to be as small as possible.
Maps in Stranded III are stored as chunks. One chunk has a size of 64x64 meters.
When storing a chunk I analyze all the terrain heights of that chunk and determine the lowest and the highest height.
Then I calculate the biggest height delta. That's simply:
delta = highest - lowest
Example: If the highest point on that chunk has a height of 5.13m and the lowest has a height of 3.02m then it would be
delta = 5.13 - 3.02 = 2.11
I decided (for now) to save the terrain with a height precision of 1cm. That's a pretty high precision in my opinion. Maybe I will reduce that a bit in future.
Based on that precision and the delta on the chunk I can determine which data type I use to store the height.
With a 1cm precision I can use a single byte (8 bit) per height value if the height delta on that chunk is 2.55m or below (a byte can store values from 0 to 255).
To do this I write the min height of the chunk and then I write the deltas of that min height and each point in the chunk as bytes.
Of course there's also a little (byte) flag which denotes that the heights of that chunk are stored in bytes. Otherwise I wouldn't know how to load it.
If the deltas are 655.35m and below I can use an unsigned short (16 bit = 2 bytes) to store the heights. In practice this is nearly always the case.
If they are above that I have to use an unsigned int (32 bit = 4 bytes). That's enough to cover even the most insane height deltas (42,949,672.95m). I don't go beyond that.
A further optimization here would be to use a 3 byte data type (24 bit, 167,772.15m) instead of directly jumping from 2 byte to 4 byte precision.
I didn't do so yet simply because this data type is not directly supported by C# and I need to implement my own read/write methods for that first. That should be quite easy though so I'll surely add it later
When testing this with real procedurally generated maps I noticed that the single byte precision was never used. This means the height delta was always above 2.55m on a 64x64 area (which isn't a big surprise at all).
Therefore I added another level of optimization: If the byte precision storage method can't be used on a chunk that chunk is split. The sub chunks will have a size of only 4x4 meters (16m²) each.
For every sub chunk I do the very same thing I did for chunks: Find min, max and delta and check which data type can be used for storing them.
Since the sub chunks are much smaller it's much more likely that bytes can be used to store the heights. This helped a lot to further reduce the map size.
Why 16m² for sub chunks? Each (sub) chunk requires 5 additional header bytes (the min height and a byte for the used data type). Therefore it doesn't make sense to make them too small.
They also can't be too large because then the height deltas would be too high for using single bytes.
I did some tests and it turned out that 16m² leads to the smallest map files.
Finally I use the gzip algorithm to compress the whole thing.
There is still a lot of room for further optimizations but I'm already quite happy with the current solution
Stay healthy!
Corona is all over the place unfortunately. I was fascinated by the numbers and at the same time frustrated that most pages did not show the information I wanted to see.
Therefore I spent some time with working on my own website related to that topic:
CoronaCurve.deIt was a fun experience! I got to know Chart.js and the CSS framwork Bulma in this little experiment - which might be useful in future.
Most other info pages improved a lot by now and provide more detailed information than I do on my little page.
I can't (and don't want to) keep up with that so I promise that I will focus more on Stranded III again
In other words: There are no plans to put more time into CoronaCurve.de!
Map Editor
Back to the actual topic of this blog: Stranded III! There's some progress with the map editor!
You can now transform the terrain with tools. This includes changing height and textures.
When changing height you can (un)tick a little checkbox which controls if the terrain colors should be automatically adjusted based on the terrain height and steepness.
I recorded a little video to demonstrate what that looks like.
Watch the map editor preview video on YouTube!Please note:
The editor is unfinished and so are the assets. Some models and textures will be changed in future. There are no LOD models yet to be displayed for far away objects. That's why stuff just disappears in this video. There are supposed to be preview images in the object list. Those are currently broken / not properly configured. The items are not properly configured in this video (ugly shiny material properties and weird colliders for physics)! Terrain normals are not calculated yet, making the terrain look very "unshaded" and flatMap Format: Storing Heights (warning: technical stuff ahead!)
I also worked on improving the map format. It's a binary format with a lot of little tweaks which help to minimize the file size of maps (and save games as well).
In these days CPUs are really fast. Therefore the bottleneck when working with files is almost always the physical storage (HDD or SSD). It's still a very slow operation to read/write data (in comparison to other tasks that happen on the CPU/GPU directly).
SSDs are much faster than HDDs for sure but there's that issue with limited write cycles. Therefore I want the map files and save games to be as small as possible.
Maps in Stranded III are stored as chunks. One chunk has a size of 64x64 meters.
When storing a chunk I analyze all the terrain heights of that chunk and determine the lowest and the highest height.
Then I calculate the biggest height delta. That's simply:
delta = highest - lowest
Example: If the highest point on that chunk has a height of 5.13m and the lowest has a height of 3.02m then it would be
delta = 5.13 - 3.02 = 2.11
I decided (for now) to save the terrain with a height precision of 1cm. That's a pretty high precision in my opinion. Maybe I will reduce that a bit in future.
Based on that precision and the delta on the chunk I can determine which data type I use to store the height.
With a 1cm precision I can use a single byte (8 bit) per height value if the height delta on that chunk is 2.55m or below (a byte can store values from 0 to 255).
To do this I write the min height of the chunk and then I write the deltas of that min height and each point in the chunk as bytes.
Of course there's also a little (byte) flag which denotes that the heights of that chunk are stored in bytes. Otherwise I wouldn't know how to load it.
If the deltas are 655.35m and below I can use an unsigned short (16 bit = 2 bytes) to store the heights. In practice this is nearly always the case.
If they are above that I have to use an unsigned int (32 bit = 4 bytes). That's enough to cover even the most insane height deltas (42,949,672.95m). I don't go beyond that.
A further optimization here would be to use a 3 byte data type (24 bit, 167,772.15m) instead of directly jumping from 2 byte to 4 byte precision.
I didn't do so yet simply because this data type is not directly supported by C# and I need to implement my own read/write methods for that first. That should be quite easy though so I'll surely add it later
When testing this with real procedurally generated maps I noticed that the single byte precision was never used. This means the height delta was always above 2.55m on a 64x64 area (which isn't a big surprise at all).
Therefore I added another level of optimization: If the byte precision storage method can't be used on a chunk that chunk is split. The sub chunks will have a size of only 4x4 meters (16m²) each.
For every sub chunk I do the very same thing I did for chunks: Find min, max and delta and check which data type can be used for storing them.
Since the sub chunks are much smaller it's much more likely that bytes can be used to store the heights. This helped a lot to further reduce the map size.
Why 16m² for sub chunks? Each (sub) chunk requires 5 additional header bytes (the min height and a byte for the used data type). Therefore it doesn't make sense to make them too small.
They also can't be too large because then the height deltas would be too high for using single bytes.
I did some tests and it turned out that 16m² leads to the smallest map files.
Finally I use the gzip algorithm to compress the whole thing.
There is still a lot of room for further optimizations but I'm already quite happy with the current solution
Stay healthy!