Page 1 of 1

OverFlowException when caching a file(ES3.LoadRawBytes)

Posted: Sun Apr 14, 2024 4:55 pm
by ColdFire
Hi Joel,
Today I found that when caching a file, the line "var bytes = new byte[stream.Length]" in ES3.LoadRawBytes() limits the maximum cached data to about 2GB (due to the array length being restricted to within the maximum value of an int), while my save files may exceed 2GB when there are 20 or more world blocks. I was planning to implement a multi-file saving tomorrow (one file per world block). However, considering the limitations of my experience as a solo developer, I'd like to hear your opinion before getting started: Is it common and acceptable to have large-capacity save files for games with procedurally generated content? Is saving a single archive in multiple files a common solution? Since the game world is randomly generated, I choose to save everything (by world block unit, saving all its child objects) to avoid creating specialized data structures and methods for saving and loading thousands of custom classes. Is this idea feasible? A 2.5GB save file can be compressed to around 400MB, so I plan to reduce memory usage with a multi-file saving and rely on file compression to reduce the size of the save files in hard drive, and keep the way of saving everything unchanged. What do you think? :)

Re: OverFlowException when caching a file(ES3.LoadRawBytes)

Posted: Mon Apr 15, 2024 1:29 pm
by Joel
Hi there,

Generally you would break your files down into smaller save files rather than having one extremely large monolithic save file, otherwise simply the operation of loading that file into memory and resaving it will be a significant bottleneck when actually you only need a small portion of the data within the file.

All the best,
Joel

Re: OverFlowException when caching a file(ES3.LoadRawBytes)

Posted: Mon Apr 15, 2024 2:30 pm
by ColdFire
Great, I will do that. :D