Gkiokan / hb-store-cdn-cli-server

HB-Store CDN Server [cli-version]
34 stars 6 forks source link

organization by folder and by type #10

Open devcrono opened 1 year ago

devcrono commented 1 year ago

@Gkiokan , I recently started using your server for a CDN on my domain, and it turned out fine, but I noticed that the pkgs ALL need to be in a common folder, I was wondering if there's no way to make one organization by paws Ex. Folder R/resident_4_cusaXXXX/xxxz.pkg Resident_4/UPDATE/xxx_v1.1.pkg In other words, inside the E folder there is the resident folder, and inside the resident folder there is the pkg of the base game, and an UPDATE folder, and inside that folder the update.pkg

Imagine that you have another game with the initial R, it would be inside the R folder etc....

Or just make 3 folders Base Update Dlcs

And everything will go to its destination folder, if base in base, if update in update and if DLC in dlc folder

Tks

Gkiokan commented 1 year ago

Hi,

from your PC you can do that actually as the Server does (should?) scan also sub folders.

But on the HB-Store there is not such "Folder Filtering" there.
The only thing I can think of would be to categorize the PKG types by Game, Update, DLC to the existing categories in the HB-Store.
Not sure if @LM could implement it this things as deep as you want with the folders, but the Categories could be done for sure.

For now all found PKGs would be assigned to the type of games if I remember right.

But currently I have not that much time, feel free to submit a PR if you want.

greetz Gkiokan

devcrono commented 1 year ago

@Gkiokan is it possible for you to add to your script, a way for me to map a google drive?, ex. I can use rclone "Z:\", map the drive on my pc and use your script to make it become a CDN, it doesn't even need to be sorted by folder, it can even be messy.

Another thing is there a way for it to update itself like every 2/3 hours and show what was just added without needing me to stop and start the script?

Gkiokan commented 1 year ago

I've just checked my code and with this it should scan all your PKG files in the given base path. Not sure if you can select a drive itself on Windows but if you can, it should work. Just give it a try. If that doesn't work symlink your Google Drive to another folder and use that as base path.

For watching changes on the given folder we need some kind of Observer / Watcher and refresh the store data for the HB-Store itself. Even if I could create a sub process that does the job on interval you would still need to relaunch the HB-Store to get the refreshed list anyway. But yea it would kill the need for restarting the server. Btw, I've that feature on my todo list already #8

devcrono commented 1 year ago

@Gkiokan I assure you that I have already tried to do this and have not been successful.

with rclone, I could map a google shared drive with millions of pkgs, using a VPS server, or try to deploy this in cloudflare.

if the script allows me to map any type of unit, physical or virtual, I believe it will be possible to do what I want.

as i said i have a google drive with many pkgs i already mapped it locally using rclone, however it is a virtual mapping. then your script does not allow me to make it the base CDN

this script allows me to map a google drive to become an idex, and still allows me to make it become a webdav.

https://github.com/ixiumu/google-drive-webdav-workers

if it is possible to use this script to get a google drive index, and then with your script transform into cdn, the hb would be completely filled with games

take a look if possible

devcrono commented 1 year ago

@Gkiokan Another thing friend, I just saw that your script creates a store.db, I thought that if I make an include with all my gdrive links in it and tell it to start, maybe it will search for everything?!!?!?!?!

problem and that if I stop, or restart the server it remakes the file.

adds the option for it to read links from the store.db itself, and ask if you want to replace the file

I will attach the store.db for better understanding file.csv

devcrono commented 1 year ago

@Gkiokan

and only this message that makes all the difference, store.db has been renewed if he didn't replace the DB, or asked if you want to renew I believe it would work.

I tried to circumvent the system and switch with the server running. without success when starting the download the app closes. when stopping and starting, or restarting the server we have a store.db has been renewed

and the modifications are overwritten

Gkiokan commented 1 year ago

Hmm interesting. I must investigate and try myself what happens when I put a shared gdrive folder as the base path.
The code does only try to read all the files that resides in that folder. Maybe there is something different with virtual folders that are not physically on your disk.

The store.db file is necessary for the HB-Store to work. It will be served though http://CDNhost/store.db and the HB-Store reads that file in order to know which files exist and where to download them. The express server that runs underneath the CLI does stream the found files in your base path.

Yes the store.db is created on startup, this is expected and required but it will be anyway implemented with the watcher when the files on your drive does change.

Did you try to put the base Path manually in the config file as you stated that you can't select it?

I quickly lookup for gdrive connection and there is a way to map the gdrive files actually. It seems there is an API Key required to authorize the CLI to gdrive with your user but that would fix the issue with gdrive at least.

But you may need some patient as I can't work on the CLI right now, has a couple stuff to finish first.

devcrono commented 1 year ago

Hmm interesting. I must investigate and try myself what happens when I put a shared gdrive folder as the base path. The code does only try to read all the files that resides in that folder. Maybe there is something different with virtual folders that are not physically on your disk.

The store.db file is necessary for the HB-Store to work. It will be served though http://CDNhost/store.db and the HB-Store reads that file in order to know which files exist and where to download them. The express server that runs underneath the CLI does stream the found files in your base path.

Yes the store.db is created on startup, this is expected and required but it will be anyway implemented with the watcher when the files on your drive does change.

Did you try to put the base Path manually in the config file as you stated that you can't select it?

I quickly lookup for gdrive connection and there is a way to map the gdrive files actually. It seems there is an API Key required to authorize the CLI to gdrive with your user but that would fix the issue with gdrive at least.

But you may need some patient as I can't work on the CLI right now, has a couple stuff to finish first.

@Gkiokan

yes I tried, manually change the store.db links, but it doesn't work, the hbstore crashes if you do that. then if I stop and restart the file and again changed then without success

about rclone mapping locally, in the case of windowsn I used raiddrive, with it the google drive becomes a physical drive, then I can map it, but I need the pkg folder and everything is messy inside

if you manage to edit the script so that with the server running you can change the store.db link and it re-reads, I believe that no matter where the link is, the hbstore will read it.

but if you manage to implement the webdav repository that I mentioned it works wonderfully, but I still couldn't make the service accounts work, so from time to time I need to go to my google panel and renew the refresh token