Find an alternative for GDrive to workaround CORS issues.

[bgiesing]

I have a feeling it's because GH Pages doesn't have the reverse proxy for Google Drive. Not sure how (or if) that could be worked around (outside of hosting it on a paid server). Any ideas?

[caseif]

The official mirror actually points to GH Pages; it just has the reverse proxy as well. I'm going to leave this open, as I'd like to find a solution that doesn't require a reverse proxy in the first place.

[itsIncept]

maybe try switching from Drive to Dropbox? You can hotlink easy by changing www. to dl. So you don't need any special setup it just starts the transfer.

[caseif]

Nah, same issue with cross-domain requests.

[caseif]

Closing for now - there doesn't appear to be a way to circumvent CORS policy and I can't fit all the current music on a free Dropbox plan (which would work otherwise).

I also tried using Google's API, but it's rate-limited and apparently the IP my university gave me already hit the limit for today, so that's not really feasible for me.

[bgiesing]

Well the main reason I think that this would be an issue is for people using their own music. As it is right now, people (most likely) either fork to make changes to contribute back, make a special variant for certain reasons (like the one by @TheNexusAvenger for getting ROBLOX sound data), or to use songs not there (possibly to make videos if they can't afford After Effects or just to listen).

So I think some suggestions to try and avoid CORS issues are ones that allow people to use their own files.

1. Use local files and a localstorage database cache

   • PROS
   • No CORS issues at all as you the files would be the user's and everything is done in real time locally already so it's definitely feasible.
   • People can use their own music without making a fork
   • Since @itsIncept site is no longer around, this could also work for those people without the hassles of maintaining the database like they did
   • CONS
   • No cloud library for people as it is now so you can't just open and let it play (you could keep both though making this not as big of a deal)
   • User would have to enter the song title and artist for each song unless you keep a local cache matching with the files (and if you rename or move a file, that would break). Maybe you could find a way to read ID3 tags to make this not as big of an issue (and still allow manual editing) but that might not be possible.

2. Well for alt. services, it sounds like you need a host with at least 5-10 GB and an API to avoid CORS? I'm not sure if this one will work but it's the only one I've found that might work:

   • Yandex.Disk: 10GB free (+512 MB for referrals up to 20GB) and here's their API docs: https://tech.yandex.com/disk/api/reference/content-docpage/

3. Google's Picker API

   • Not sure if this falls under the same rate-limiting as the Drive API but this would be an alternative to local files while still allowing people to pick there own files from Drive (again avoiding the main reason people might fork and would need to set the reverse proxy up)
   • https://developers.google.com/picker/

[caseif]

Reopening so I can try to play with these ideas after finals.

[itsIncept]

I think the best way would be to have the user pick the song using a file input, then it loads the info from ID3 tags (artist/title/genre), if none are present (or missing some) it will then prompt for more info and then once it has collected the mp3 and info it can store it to a local database. This would make it easy to code unrestricted editing and playlists as well.

I believe localstorage has a file size limit though, so might want to look into that or try an offline javascript database system that supports text and files (if that exists).

been wanting local mode for ages but never could get it to work because the whole thing runs on page load so it'd take a bit of restructuring and Max knows the code better than me so yeah hopefully he can figure it out.

If we can get this to work it'd be great really because then everyone can just run their own music locally and it cuts down on your servers stress as its not grabbing the CSV every time and removes file uploading from the equation.

Why should users have to upload to a site just from them to redownload it again? (even on refresh due to drives anti-caching header) kinda silly if you think about it.

[bgiesing]

@itsIncept Yeah it solves many issues.

1. No potential copyright issues of hosting paid music (as it is now or would be even worse with uploads)
2. No serverside at all (if it was restructured like this it could possibly run completely offline so you could either go to a website like now or just open the HTML file on your computer)
3. LocalStorage does have an approx. 5-10 MB limit so a offline database would be better. Maybe something as simple as a JSON file with paths to files might work? Yeah that would mean the file if moved would no longer be there but I'm sure the app could ask for an updated location if necessary.
4. Maybe this offline JavaScript database would work? https://pouchdb.com/

Also @caseif on a side note (probably should be a different issue), considering Monstercat no longer uses genre colors and the space background, maybe they should be removed or be adjustable as an option?

[itsIncept]

It used to run like that with mp3 files on the server but obvs moved it to other hosting so server didnt stress and copyright stuff ..but it caused all this cors issues and upload/download sillyness so local is the way to go

I think the referencing of local files is fine, not many people move their music directory, especially iTunes stuff. You can easily store something like 'file://blah.mp3' and yeah just prompt if they try to load something from the list thats missing broken

Re genre: Keep it as a legacy option, as we worked so hard on it lol but yeah probz have to have a field for pictures too.

[caseif]

Regarding local storage: I might implement that in addition to the way it currently works, so maybe it'll default to the current behavior but also include a /custom slug or something where it allows custom upload.

Regarding genre colors: probably allow background images to be set, in which case it'll use the white color. Otherwise, it should default to a genre color.

As I mentioned, I'll probably try to work on this over winter break; I definitely want to expand the project this way when I'm able to.

[bgiesing]

@caseif Yeah that's probably the best solution there as while local would be the better thing overall, it does require some more setup so there still should be a "library" like their currently is.

Also I went ahead and renamed the issue as this issue is more about CORS than GitHub pages.

[itsIncept]

I have a load of time rn so I'm gonna see if I can dodgily implement the local idea with a simple GUI for input and playlists even

[itsIncept]

Okay so I coded an offline music database thing http://incept.cc/dev/ took forever. It uses an ID3 library to quickly get song info, Base64 encoding/decoding for Mp3's/Images and IndexedDB to store all the data. I think we could implement it fairly well as IndexedDB sticks to the domain, is persistent even on browser close and has unlimited storage unlike all the other alternatives (apart from the hard drive limits of cause).

Would just need to figure out how to use base64 with web audio/convert to something usable and use the track info from the database instead of a file.

which is kinda beyond me cos yeah the vis code is pretty rough.

[bgiesing]

@itsIncept Just tried it on both Edge and Chrome and it seems to work fairly well outside a couple of issues (to be expected for something whipped up this fast)

1. DO NOT load FLAC files. I accidentally did instead of MP3 and it instantly froze my computer then forced a restart. I'm guessing this might be because the ID3 library as according to the official FLAC/Xiph website "some implementations may not even be able to decode a FLAC file with ID3 tags". So for a finished product, there would need to be some detection of filetypes to prevent people from loading it.
2. The "View All Records button in Edge doesn't work at all.

[itsIncept]

@bgiesing lolrip I forgot to add the accept attribute I guess. I believe that's probably due to IndexedDB being a relatively new thing that only started to pick up in 2016 so edge might not support it yet.

[itsIncept]

Uh so I just loaded a flac file fine, it played and everything but the id3's didn't go through so I assume you're right there with it being the issue.

[bgiesing]

@itsIncept According to Microsoft's dev site, Edge does so IDK what happened there https://developer.microsoft.com/en-us/microsoft-edge/platform/documentation/dev-guide/storage/IndexedDB/

Well as for the FLAC crash and reboot, my computer is a shitty Celeron so maybe the ID3 could have caused the browser freeze but the Celeron couldn't take it and rebooted the computer. Essentially, I think that might have been a rare occurrence.

[itsIncept]

the console error is something completely different to db actually so yeah not sure what's going on there. It's just a prototype anyway and it works so when it's actually implemented that's when we can start being careful with it aha.

[itsIncept]

Managed to use Dexie.js (IndexedDB API Wrapper) to get the idea working even better plus edge support. Can store mp3's as blob now instead of converting to base64 (still have to do that for the image though because id3 data is different). There's not a lot of validation or styling because I wanted to keep the code minimal and clean but this should be working fine: http://codepen.io/Incept/pen/XNOReE

[yeSpud]

So, I realize its been a while, but after reading the above, I gotta ask: is this project at an official stand-still until a workaround is properly implemented, & someone commits the changes?

[itsIncept]

We developed a localdatabase method for the Trap Nation vis which works well and is being ported over to vis reborn eventually.