gildas-lormeau / SingleFile

Web Extension for saving a faithful copy of a complete web page in a single HTML file
GNU Affero General Public License v3.0
15.76k stars 1.02k forks source link

more remote destinations choices #1308

Open paponius opened 1 year ago

paponius commented 1 year ago

Is your feature request related to a problem? Please describe. sort of. google drive feature not working, box.net dropping DAV support

Describe the solution you'd like Another protocol/cloud. like FTP/dropbox. I do save a web page, attach the file to email, use plugin in Thunderbird to convert the attachment to dropbox and send. I expect clickable link to show the page on the other end. I wanted to simplify this process. maybe even an option to save to cloud and open email to send the link. (mailto protocol) I usually use dropbox for this, but I guess gdrive is fine too, if it worked.

cheers

gildas-lormeau commented 1 year ago

From a technical point of view, it's impossible to use FTP in a Web Extension.

It seems Dropbox could be supported, see https://www.dropbox.com/developers/documentation/http/documentation. I need to do some tests to check if it's supported in a Web Extension.

FYI, you could maybe use SingleFile Companion Lite to save pages in a synchronized folder on your disk as a workaround. See https://github.com/gildas-lormeau/single-file-companion-lite.

gildas-lormeau commented 1 year ago

I added the support of Dropbox as a new destination. The feature will be available in the next version.

gildas-lormeau commented 12 months ago

@paponius Out of curiosity, did you try the new destination for Dropbox?

paponius commented 11 months ago

image As I understand it, Dropbox must review your app.

Applying for production status

If you don't request the production status from the Dropbox, the app will have a limit of 500 users. That I already hit the threshold means there was quite a demand for Dropbox in your app. And that you have an impressive user-base.

You can now request for additional "test users" to solve the limit immediately but temporarily. Enable additional users on the link above. And also request for the production status. When you'll receive it, change the API keys in your production version to those they send you.

For the Google Drive,

It works for me now, but I had to switch off/on the plugin, it was saving to disk first time after changing to GDrive.

Also, a custom-named directory would be nice, now it's saving to the root dir of GDrive. And something is not completely OK with GDrive app connection, maybe similar problem as the Dropbox? To verify the app? image

Anyway, I don't use GDrive for this, as it will not show you the html as an html, but as a plain text source. Dropbox does it somewhat better, in a frame, but the file can be read OK. Maybe I'll set dropbox or GDrive sync on NAS to sync to a web accessible dir. But that's not a solution for all users. What about using your helper app to upload to own web server using FTP?

gildas-lormeau commented 11 months ago

Thank you for the feedback, I've requested the production status to Dropbox.

You can add a path into the filename template, e.g. Archives/%if-empty<{page-title}|No title> ({date-locale} {time-locale}).{filename-extension}.

The companion could indeed implement a FTP client.

jave commented 11 months ago

I would like the option to have a separate download directory, rather than the browser default download directory. In my case I would like to use syncthing to put my singlefiles on a server, and the browser default dir is too unorganized.

jave commented 11 months ago

I noticed https://github.com/gildas-lormeau/single-file-companion-lite, i will try that

paponius commented 11 months ago

That's a browser limitation. WebExtension does not have an access to the OS filesystem.

If you have full access to the server where you want SingleFile to put saved files to, explore also the WebDAV option.
To put HTML files on a web server is after all what the WebDAV is originally for.

QNAP:
https://www.qnap.com/en/how-to/tutorial/article/accessing-your-qnap-nas-remotely-with-webdav OR https://docs.qnap.com/operating-system/qts/5.0.x/en-us/configuring-webdav-settings-CDDF133D.html Synology:
https://kb.synology.com/tr-tr/DSM/help/WebDAVServer/webdav_server?version=7

@jave

ryanwwest commented 11 months ago

Would Samba be easily supported? Similar to how WebDAV is already supported. Not sure if it would also be impossible like FTP.

paponius commented 11 months ago

FTP is not "impossible" per se.
It may be possible in passive mode using XHR. I did not find a definitive reason it can't be done.
And it is possible using a proxy. There was a project, https://blog.dotnetframework.org/2016/06/07/upload-ftp-using-javascript-only/ both links are long invalid, but there is no reason such proxy could not be created on some cloud site.

Samba with JS is even less debated and explored. There are Node.JS libraries for both Samba and FTP, but nothing to be readily used in browser's JS.

I would say, even if it may be theoretically possible, it would be a big project by itself. And not something I would expect to be developed just for the purpose of adding such function to SingleFile plugin.

The companion app could add Samba, but I am not sure how big demand would be for such feature and how big a priority it would be.
Maybe there is an easier solution how to satisfy such specific requests. Samba, FTP, or even Send by e-mail, or post to twitter...

An option to run a command after a page is saved. With possibility to use placeholders like in output file name.
Then a script can take the saved file and send it by Samba somewhere, or on Windows by a simple powershell.exe -c Copy-Item ... one-liner command.

ryanwwest commented 11 months ago

@paponius Thanks for the reply. That is fine, adding Samba isn't essential especially since we already have WebDAV which likely can accomplish a similar purpose most of the time. More variety is nice but not needed in this particular case. If it becomes as simple as WebDAV was to add in the future, maybe then it can be reconsidered.

yooper commented 6 months ago

@gildas-lormeau , great product you made. I would like to POST the captured content and its originating URL to an API endpoint. The authentication for sending to the endpoint would be an Authorization Bearer token. If I were to code this up and provide a PR would you except it into SingleFile?

Thank you

gildas-lormeau commented 6 months ago

@yooper I guess the implementation would be very similar to the one used for github, see https://github.com/gildas-lormeau/SingleFile/blob/e57af7306a2002ffea21e8036269bc543d3b94ec/src/lib/github/github.js. What name would be given to this kind of API in SingleFile? Is it standardized or very easy to describe in a specification document?

yooper commented 6 months ago

Hello @gildas-lormeau , Yes, it would be very similar to the Github destination implementation.

The name for the destination would be REST API Endpoint. The user would have to provide:

  1. REST API Url -> the url where the data will be sent to
  2. HTTP Method -> POST or PUT
  3. Auth Bearer Token -> the token
  4. File Field Name -> The field name the file will be sent as
  5. Url Field Name -> The field name of the URL will be sent as

Configuration values that are hard-set

  1. Content-Type -> "multipart/form-data"
  2. Http Success Code -> status code 200 or 201 is a success, otherwise an error
gildas-lormeau commented 6 months ago

Thank you for the info. Could you give me more details about 4 and 5? What are field names?

yooper commented 6 months ago

Sure.

The Content-Type header in an HTTP POST/PUT request indicates the type of data in the request body.

HTTP status codes are three-digit numbers that a server sends to a browser's request to indicate if a request can be fulfilled. This is useful for displaying an error if the REST API endpoint is not working.

Hope that helps.

gildas-lormeau commented 6 months ago

Thank you but I would like to know what "File Field Name" (4) and "Url Field Name" (5) are for.

yooper commented 6 months ago

Sorry about that..

To provide versality so that the field names can get mapped to the whatever the field names are in the end point. For example, I have an API endpoint that expects the name of the file attribute as ContentData and the url to be sent as ContentUrl. Another user might have different set of expected fields in their endpoint.

Thank you,

gildas-lormeau commented 6 months ago

Thank you, so they woule the field names of the form. Your proposal serms OK to me. Maybe the word "form" should be in the name, e.g. 'REST Form API Endpoint'

yooper commented 6 months ago

Hey @gildas-lormeau , I submitted a PR for pushing saves to a REST API endpoint. Please let me know if you have any questions. Really looking forward to having that feature enabled in SingleFile.