I want to upload this thing I just downloaded off the internet
or i already did?
I want to discover stuff in object store
I want to upload stuff I found on a hard drive (noting that MC doesn't have a DB)
I want to get a cached version of a URL
I want to list files with a given metadata filters
I want to list a directory from a HD
POST /upload - this is a convenience function etc which basically if they know some stuff about a file it either sends them to existing or creates stuff to allow an actual upload
mtime (from source) - datetime optional
URL - for file on a host file:///; or mc:///; for ytdl it is the webpage - 2047 - mandatory
direct T/F - i.e. if i do wget url will i get exactly this or do i need to finnegle this , optional assume true
Metadata JSON
ytdl - partial (below) - just those things inherent to that URL or referrer
lm backend
referer info (optional) - URL_referer - ytdl URL or actual referrer or some ID of disk (i.e. HDD it was found on with mc)
i.e. ytdl 'webpage_url' - url of page itself - file referer
metadata intrinsic
exif
filename
user - 16/32 - if no authentication
ul_sw - 16/32
ul_host -16/32
partial i.e. this is just a partial rep of URL, etc
Bucket name -varchar 63
Size
hash
MIME - mime type 255
return -
file - file object including object object
upload info (or NONE if existing object)
path to upload to in obj store
path to post to to complete
download info (or NONE if not yet)
easy link to obj store
POST /upload/objid
complete=True
(or just PUT to /object???)
POST /file (dynamic create object)
POST /object (discovered object)
GET files
search by metadata, prefix, bucket, etc
single item lookup by hash, name, etc to get real location
redirect to real location
note
MC doesn't have long term storage, but we can put some info on the parent Tree object in file metadata
originally we had seperate url and url_referer... however
this makes indexing etc complicated, two fields instead of one, so merging and adding boolean direct T/F field
ytdl use case URL is the 'webpage URL', direct is F... any info on actual DL can be in metadata
general web case, it is direct URL, direct T, any referer info in metadata
MC - we will likely concoct a url-style identifier for MC, or fall back on upload_host file:///; direct would be T; some MC data in metadata
twch - partial T direct F, see mtime, and metadata
Based on #4 #9
POST /upload - this is a convenience function etc which basically if they know some stuff about a file it either sends them to existing or creates stuff to allow an actual upload
return -
POST /upload/objid
POST /file (dynamic create object) POST /object (discovered object) GET files
search by metadata, prefix, bucket, etc single item lookup by hash, name, etc to get real location redirect to real location
note