Improving Performance Of Saving Many Files At Once

[FreedTerror]

I'm looking to improve the performance of saving many files at once.
My current method for doing this is causing a noticeable pause when trying to save 10 - 20 files.
Here's a test script that sort of replicates what I'm trying to do.

Code: Select all

using UnityEngine;

public class EasySave3PreformanceTest : MonoBehaviour
{
    [Range(1, 50)]
    public int numberOfSaves;
    public bool saveToCache;
    public bool loadFromCache;
    public int[] myInts;

    // Update is called once per frame
    void Update()
    {
        if (Input.GetKeyDown(KeyCode.Space))
        {
            SaveTest();
        }

        if (Input.GetKeyDown(KeyCode.N))
        {
            LoadTest();
        }
    }

    private void SaveTest()
    {
        for (int i = 0; i < numberOfSaves; i++)
        {
            if (saveToCache == false)
            {
                ES3.Save("myInt", Random.Range(1, 101), "Preformance Test/Int Save Data " + i + ".es3");
            }
            else
            {
                // Create an ES3Settings to save to and load from the cache.
                var settings = new ES3Settings(ES3.Location.Cache);

                ES3.Save("myInt" + i, Random.Range(1, 101), settings);
            }
        }
    }

    private void LoadTest()
    {
        if (loadFromCache == false)
        {        
            string[] fileNames = ES3.GetFiles("Preformance Test");

            myInts = new int[fileNames.Length];

            for (int i = 0; i < fileNames.Length; i++)
            {
                myInts[i] = ES3.Load<int>("myInt", "Preformance Test/" + fileNames[i]);
            }
        }
        else
        {
            // Create an ES3Settings to save to and load from the cache.
            var settings = new ES3Settings(ES3.Location.Cache);

            myInts = new int[numberOfSaves];
            for (int i = 0; i < numberOfSaves; i++)
            {
                myInts[i] = ES3.Load("myInt" + i, 0, settings);
            }
        }
    }
}

Is there a way to improve performance of saving many files while also creating the needed save files in the Save folder?
I tried caching, but was unable to create the needed save files in the Save folder.

[Joel]

Hi there,

There appear to be some mistakes in your code.

Your code doesn't appear to be calling ES3.CacheFile or ES3.StoreCachedFile as in the docs:

https://docs.moodkie.com/easy-save-3/es ... rformance/

Your ES3.Save and Load calls for cache also don't appear to be specifying a filename, so they still all save to the default file.

However, note that caching will only improve performance of writing multiple keys to a file, not writing multiple files. If you're only writing a single key to each file then the performance benefit will be small.

In your case you're writing multiple files with a single key, so the performance overhead will likely be the file IO operation. The performance of IO operations are limited by your device, so there would be no way to improve this.

All the best,
Joel

[FreedTerror]

You're right I missed some things in the cache code.
I tested this refined version and it seemed to work.
I got no performance gains with the file writing using the cache.

Code: Select all

using UnityEngine;

public class EasySave3PreformanceTest : MonoBehaviour
{
    [Range(1, 50)]
    [SerializeField]
    private int numberOfSaves;
    [SerializeField]
    private bool useCache;
    [SerializeField]
    private bool save;
    [SerializeField]
    private bool load;
    [SerializeField]
    private int[] myInts;

    // Update is called once per frame
    void Update()
    {
        if (save == true)
        {
            Save();
        }

        if (load == true)
        {
            Load();
        }
    }

    private void Save()
    {
        save = false;

        if (useCache == false)
        {
            for (int i = 0; i < numberOfSaves; i++)
            {
                ES3.Save("myInt", Random.Range(1, 101), "ES3 Preformance Test/Not Cached/MyFile " + i + ".es3");
            }
        }
        else
        {
            for (int i = 0; i < numberOfSaves; i++)
            {
                // Cache a local file and load from it.
                ES3.CacheFile("ES3 Preformance Test/Cached/MyFile " + i + ".es3");

                // Create an ES3Settings to load from cache.
                var settings = new ES3Settings("ES3 Preformance Test/Cached/MyFile " + i + ".es3", ES3.Location.Cache);

                // Save to the cached file.
                ES3.Save("myInt", Random.Range(1, 101), settings);

                // Store the default cached file to a permanent local file.
                ES3.StoreCachedFile("ES3 Preformance Test/Cached/MyFile " + i + ".es3");
            }
        }
    }

    private void Load()
    {
        load = false;

        if (useCache == false)
        {        
            string[] fileNames = ES3.GetFiles("ES3 Preformance Test/Not Cached/");

            myInts = new int[fileNames.Length];

            for (int i = 0; i < fileNames.Length; i++)
            {
                myInts[i] = ES3.Load<int>("myInt", "ES3 Preformance Test/Not Cached/" + fileNames[i]);
            }
        }
        else
        {
            string[] fileNames = ES3.GetFiles("ES3 Preformance Test/Cached/");

            myInts = new int[fileNames.Length];

            for (int i = 0; i < numberOfSaves; i++)
            {            
                // Cache a local file and load from it.
                ES3.CacheFile("ES3 Preformance Test/Cached/MyFile " + i + ".es3");

                // Create an ES3Settings to load from cache.
                var settings = new ES3Settings("ES3 Preformance Test/Cached/MyFile " + i + ".es3", ES3.Location.Cache);

                // Load from the cached file.
                myInts[i] = ES3.Load("myInt", 0, settings);
            }
        }
    }
}

I don't want a noticeable pause when saving, so I'll just stick to writing single files at a time. I got the best results that way.

Thanks for the support! Easy Save 3 has been great to work with so far!