ygojson / ygojson-tools

Tooling for the ygojson project
MIT License
1 stars 0 forks source link

Evaluate multi-thread processing #168

Open magicDGS opened 5 months ago

magicDGS commented 5 months ago

After #167, we are using mutiny to both access the yugipedia-API and the conversion to the property map. Nevertheless, I believe that this is not multi-thread still, just not-blocking the main thread.

We can most likely schedule the async execution on a thread-pool, using even virtual-threads, to speed up processing. This is not a must yet, but can benefit greatly the pipeline for the initial data collection (that will be quite important for the first iterations of the data until we estabilize our models).