Image Embedding and Encrypted Attachments

[MistMachine]

Editor's note: This issue was originally posted to address a workaround to functionality that is no longer supported. The issue then spurred a discussion around the feasibility of encrypted attachments in Standard Notes.

October 2018 Update: FileSafe is now available, and allows encrypted attachments. (No inline image support yet though.)

Original Issue

Sooo.... I am not sure if there are any plans to improve implementation of embedding images but I figured a not so simple work around with the Markdown Editors that could be implemeneted as an extension or integrated into the file attacher extension. 1) Attach the the desired image to the documents with the file attacher extension 2) From the action bar, drop down to Manage Attachements and copy the link of the opened page 3) From this link you need to replace some parts of the link to navigate to your attached image https://standardnotes.org/ext/file_attacher/b14e81ca-fa45-4a9e-abda-f9dfe567ab78/files?key=*******************your_personal_encryption_key********************&item_uuid=***********specific_document*********** to https://standardnotes.org/ext/file_attacher/b14e81ca-fa45-4a9e-abda-f9dfe567ab78/download?key=*******************your_personal_encryption_key********************&file_path=/file_attacher/***********specific_document***********/name_of_image.png (I figured this out by using action to download the file I attached to the document and quickly copying the link from the tab that it opens in my Chrome Browser, so that method also works) Then input this link to the desired format for embedding images The resulting code will look like < img src = "https://standardnotes.org/ext/file_attacher/b14e81ca-fa45-4a9e-abda-f9dfe567ab78/download?key=*******************your_personal_encryption_key********************&file_path=/file_attacher/***********specific_document***********/name_of_image.png"> Anyways, I am posting this hoping to see this tedious work around that already utilizes existing features to embed your own images to your documents. I would try to send a push myself but I am not very experience at making and designing software Loving this product by the way. So glad I bought a sub.

[moughxyz]

@PopMachine wow, hell of a workaround :) Trust me, there's nothing I want more than to have inline attachments and a built-in attachments interface, but that stuff's hard and would detract from the main product right now, so it's postponed into the future. But, glad to see you have an interim solution.

[fluffymadness]

I'd would be awesome to have inline images e.g via drag and drop in the future. Looking forward to it. I love to have my notes at one place. And right now if I want to write bigger notes with embedded images and tables, eg when I write some documention of something, I still have to use word or libre office and save it someplace else then my other notes.

[dreamflasher]

@mobitar What's your estimate on the probability that this feature (inline images) will be implemented during this year (2018)? To me this is a crucial feature and currently blocking me from switching to standard notes.

[moughxyz]

Very low chance of this year. Probably 2019.

[willc]

I too would like to add my voice in support of supporting inline images. This is the crucial feature preventing me from switching to Standard Notes.

[ghost]

Really hope this this feature is not implemented. SN to me is a simple note taking app not another place I have photos hanging around. Inline images is feature bloat IMO.

[dreamflasher]

@gelviis You fundamentally don't understand the design and principles of standard notes. Did you have a look at extensions and how features are added optionally? I'd recommend reading up on it.

[ghost]

@DreamFlasher that's a bit harsh...

[willc]

A bit harsh, perhaps, but not entirely incorrect. I don't think anyone is intending this to be a photo storage app, as there are plenty of better solutions for that. Rather, being able to take screenshots and embed them inline would be invaluable for many people, including developers, designers, pentesters, or engineers.

You get out of SN what you put into it, so if you don't want photos "hanging around," don't put them in there.

[moughxyz]

All valid points :) It would definitely be an extension if it were to happen, and not native. Here's why I'm resisting working on this for now:

1. Encryption. Photos would need to be encrypted. There's no mechanism right now to share your encryption keys with extensions. So somehow, the extension would need to encrypt photos. This would probably require you inputting a custom key. Then, what about if you want to change the key? What about if you forget the key? A really challenging problem.
2. Mobile. Extensions are not available on mobile. We already get a barrage of "when will editors make it to mobile?" I would not be able to handle another influx of "why can't I view my attachments on mobile?"
3. Storage and limits. Where would the files be stored? What limitations would we have to put in place? Would it be another paid tier on top of Extended, or part of the current offering? Most likely, this will involve a whole other level of costs, meaning we'd have to modify Extended to add storage tiers. A very hard problem.
4. Support. Given the engineering feats required for this problem, bugs and issues are bound to occur. I can't currently afford more problems. Focus helps keep a more peaceful life and business.

With all these reasons combined, you can see why I'm not rushing to create this. I still want to focus on the fundamentals. We're a very, very far way from this.

[ghost]

@mobitar spot on. Thanks for clarifying.

[fqure]

Maybe we could consider a survey on your above 4 points. On item #1, I for one wouldn't need the photo/image itself encrypted, just the link (which by being inside the note itself would be encrypted). So maybe this beta v 0.1 extension would have a disclaimer that the standard server images folder is not encrypted?

[willc]

I don't care about mobile, personally, though I can see that others might. Since images don't work well in mobile, leave them out of it. Support them in the browser/desktop. I don't need them to be encrypted either. I just need to drop images into my notes -- especially screenshots. I don't care where they are stored, though if they are stored in my Google Drive by association, that'd be fine. And they'd inherit encryption that way, no?

[moughxyz]

I don't need them to be encrypted either.

The thing is, if we're going to spend all this time building out a file attachment service, knowing full well that one day we're going to need to make it encrypted, it really makes no sense to spend the time now. We're a privacy-focused company, so if we put all these caveats like "this is private, but this is not", then we really weaken our trust with users. I'd rather users automatically know that everything is private, rather than, "I wonder if this part of the application is private?"

Also, I'm not under any rush, so would prefer to do it the right away, rather than the quick way. If users turn away from SN because it doesn't support images, it's not a huge issue. We select for users who want simplicity anyway.

[fqure]

Agree about simplicity. There are 2 models for assets (images/video) to consider:

1. Encapsulated- The entire asset would need to be encrypted. I think you explained this above clearly
2. Externally linked– An extension: a) The main SN text area would need to be a drag and drop that creates a link. b) An external API call for write storage (Google Drive, Dropbox, Box, AWS etc) to then externally store the asset c) Stating the obvious, but the transport would need to be SSL

If the debate is 'everything needs to be encrypted' then option 1 would incur very large files or sql blobs-- a bloated SN. Large sizes also reduces portability and transport. Item 2. is currently already in the core as text only links.

Both scenarios would write the API as long string files names. Option 1 could leave it unencrypted and consider a script to post encrypt at a later stage of development (we're just re-ordering bits assigned to a key pair). In Option 2 scenario, encryption would be left to the external storage service, not Standard Server.

So this is really a debate of what system services a user's privacy. Is SN part of a user's overall privacy or does it need to be the entire stack for privacy? Also, if there were a breach of an external storage system, is the concern that SN would be blamed?

[willc]

If users turn away from SN because it doesn't support images, it's not a huge issue. We select for users who want simplicity anyway.

As a user who appreciates simplicity, this seems like a simple feature. Drag/drop or copy/paste an image into your note. The complexity is on the back end, it would seem.

[moughxyz]

would incur very large files or sql blobs-- a bloated SN

We usually refer to bloat in terms of experience and featureset, not necessarily file sizes. If we were to use your Dropbox/Google Drive, and you wanted the images to be embedded in the editor, you would have to make the attachments publicly accessible via that link. It's just a privacy nightmare. Again, would rather do it right, than do it at all.

this seems like a simple feature

There's no such thing as a simple feature ;)

Every feature is a marriage proposal.

[dreamflasher]

My expectation from a user's perspective is that everything I put in SN is encrypted. So I think encrypted attachments is a necessary precondition.

[rrggrr]

@mobitar

Sensitive to your concerns and with an eye toward making this as painless to implement as possible... perhaps the simple approach is this:

1. Pass the image/attachment - unencrypted - to MYSQL on insert.
2. Encrypt using mysql's AES_ENCRYPT() function, and store as a longblob.
3. Offer the service only to users hosting their own DB (eg. Heroku is dead simple to implement).

For users who need images or other attachments its trivial to implement. I can't see much impact on your infrastructure, but I admit I've not carefully reviewed the architecture.

Hope this helps.

[moughxyz]

If we were to offer this, we definitely wouldn't alter our server infrastructure for it. We like to go in the way of extensions, not altering our sacred, working code ;)

But, I think I've already decided the architecture we'll be going with is offering encrypted attachments hosted by the user's own cloud provider (Dropbox, Google Drive, etc), rather than creating our own cloud/file infrastructure, which gives me anxiety just thinking about.

[dreamflasher]

That's very reasonable. I've been thinking about that in more general terms before: Why do you actually want to handle the whole device syncing infrastructure and encryption at all (even for text notes)? Wouldn't it have saved you a lot of work if you would have built upon https://nextcloud.com/endtoend/ ?

[moughxyz]

Just simplicity of use. It's not that bad. Would also rather not depend on another company.

[dreamflasher]

Inventing your own crypto sounds not that simple at all... This always rings for me bells; "shouldn't there already be something which gives me end to end encryption for files?" (needn't be a cloud hosting service, could also have been something like truecrypt, there are for sure encrypted file system frameworks out there) I mean, you depend on other software anyways (packages, your os...).

[moughxyz]

It's already built..what are we even debating ;)

[dreamflasher]

That's not an argument. And I am not debating ;) I was just curious -- because it doesn't make sense to me :) And even if it's already built: That's called refactoring ;)

[timmyomahony]

I would love to move over to SN but no image support is a deal breaker. I use Quiver on Mac to take notes and the ability to drop images into notes is one of its best feature. I'd recommend checking it out to see how it deals with note taking. It allows you to add section to a note, with each section having a different type: image, markdown, code, rich text etc. This allows you to build notes up with different data types.

[mlcloudsec]

+1

[WTurunen]

Only thing holding me back from Standard Notes is attaching encrypted images/files to notes, so +1 from here.

[dreamflasher]

This could actually be solved via Data URIs: http://en.wikipedia.org/wiki/Data_URI_scheme This wouldn't require any additional encryption for files and for images I think this would actually be fairly clean.

It can also be used for including any file, but the problem with it is that it's a lot harder to implement a feature of "open the file, edit the file, on save update the note with the updated file". But it's possible to include any file: https://stackoverflow.com/questions/3916191/download-data-url-file

I would consider it fair to let users pay for storage above a threshold.

[moughxyz]

This is not very practical with any image larger than a small icon. You'll quickly run into syncing bottlenecks and app performance issues.

[dreamflasher]

That's unfortunate. Do you sync deltas? Then I don't see the syncing issues.

[moughxyz]

No, tough (infeasible?) to do encrypted deltas.

[rrggrr]

How about this... Perhaps SN becomes a Heroku Ecosystem Partner. The exponential benefit being the following:

• Encrypted file attachments via mySQL longblob AES encryption at database.
• Extension-based revenue model remains untouched.
• New user adoption/exposure by embracing the Heroku platform.
• Ease of integration with other ecosystem components.

[moughxyz]

The important thing to remember is that there's a very clear distinction between Standard Notes the app, and Standard File the server. SF is the backend, and has a very simple and stringent protocol whose main tenet is simplicity and lack of growth. And SF has hardly changed at all over the last two years. We like to look at SF as "not ours to change", so, there are no custom server elements with Standard Notes. Any extension we build, even server side functionality like Dropbox backups, do not modify our server, but create separate architectures elsewhere that communicate with the base, unmodified server.

All that to say, this won't be as simple as "creating a database column that allows longblob encrypted storage." Instead, we'd have to create an entirely separate server architecture that communicates using the SF protocol of simple object-based items. Anything is possible with this architecture, but it must be done in certain ways.

SF deals with items that carry arbitrary data. For encrypted attachments, here's the steps we would need to take:

1. Create a UI modal extension (similar to CloudLink) for Standard Notes that allows the user to sign up for a separate file-hosting account (by Standard Notes, but using a different server than the one used for your SN data). This component would need to also allow setting up an encryption key.
2. Create another UI component extension that sits in the editor area (similar to the Action Bar) that allows dragging and dropping files, and retrieving and displaying associated files for that note. This extension would handle encrypting and decrypting, and would thus need to find a way to communicate with the first extension to retrieve the keys.
3. Create an independent server architecture that can retrieve and upload files to external cloud providers like Dropbox, Google Drive, OneDrive, NextCloud, etc (we wouldn't be offering our own storage; way too complex and out of scope for us.).

So, it's not impossible, but also not trivial. I'm open to it, but timing is important. I'm in no rush here. We're trying to be a sane, healthy company that is never in over our heads ;)

[dreamflasher]

Well, if you are thinking about creating a new server architecture anyways, I still believe that standard notes should leave the whole syncing and encryption part to other software that is specialized on it. And instead focus on the notes part of it. One simple way of organizing things would be to have all notes in single files (or in a db, whatever is more convenient) and let https://librevault.com/ or https://nextcloud.com/ handle syncing and encryption. (Or reuse some of their code, or https://turtlapp.com) Funnily enough while searching for solutions to this I found: https://joplin.cozic.net/ -- they do the e2e and leave the sync to external services, which is kinda nice because then people can use dropbox etc. (but still I wouldn't like to do the e2e myself and leave that for an external software too).

[RemiDesgrange]

Don't want to re-start a flame war but all I can see here are complicated solutions :

• external products
• blob in db (I though this debate was closed since S3 but I was wrong).

What I was thinking about is much much simpler : storing image as base64. It already work with markdown editor:

![Hello World](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEYAAAAUCAAAAAAVAxSkAAABrUlEQVQ4y+3TPUvDQBgH8OdDOGa+oUMgk2MpdHIIgpSUiqC0OKirgxYX8QVFRQRpBRF8KShqLbgIYkUEteCgFVuqUEVxEIkvJFhae3m8S2KbSkcFBw9yHP88+eXucgH8kQZ/jSm4VDaIy9RKCpKac9NKgU4uEJNwhHhK3qvPBVO8rxRWmFXPF+NSM1KVMbwriAMwhDgVcrxeMZm85GR0PhvGJAAmyozJsbsxgNEir4iEjIK0SYqGd8sOR3rJAGN2BCEkOxhxMhpd8Mk0CXtZacxi1hr20mI/rzgnxayoidevcGuHXTC/q6QuYSMt1jC+gBIiMg12v2vb5NlklChiWnhmFZpwvxDGzuUzV8kOg+N8UUvNBp64vy9q3UN7gDXhwWLY2nMC3zRDibfsY7wjEkY79CdMZhrxSqqzxf4ZRPXwzWJirMicDa5KwiPeARygHXKNMQHEy3rMopDR20XNZGbJzUtrwDC/KshlLDWyqdmhxZzCsdYmf2fWZPoxCEDyfIvdtNQH0PRkH6Q51g8rFO3Qzxh2LbItcDCOpmuOsV7ntNaERe3v/lP/zO8yn4N+yNPrekmPAAAAAElFTkSuQmCC)

and should work with plus editor (just replace the markdown with HTML). The problem here is readability. I don't know if standard notes have a size limit for notes ? because with this solution it can be a problem.

Here are though about integrating base64 images:

• we'll need an image converter, you import/drag'n'drop image in standard notes and it convert it in base64 (not hard)
• you can use footnotes to regain readability like:

 some text

![Hello World][helloworld]

some text

[helloworld]: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEYAAAAUCAAAAAAVAxSkAAABrUlEQVQ4y+3TPUvDQBgH8OdDOGa+oUMgk2MpdHIIgpSUiqC0OKirgxYX8QVFRQRpBRF8KShqLbgIYkUEteCgFVuqUEVxEIkvJFhae3m8S2KbSkcFBw9yHP88+eXucgH8kQZ/jSm4VDaIy9RKCpKac9NKgU4uEJNwhHhK3qvPBVO8rxRWmFXPF+NSM1KVMbwriAMwhDgVcrxeMZm85GR0PhvGJAAmyozJsbsxgNEir4iEjIK0SYqGd8sOR3rJAGN2BCEkOxhxMhpd8Mk0CXtZacxi1hr20mI/rzgnxayoidevcGuHXTC/q6QuYSMt1jC+gBIiMg12v2vb5NlklChiWnhmFZpwvxDGzuUzV8kOg+N8UUvNBp64vy9q3UN7gDXhwWLY2nMC3zRDibfsY7wjEkY79CdMZhrxSqqzxf4ZRPXwzWJirMicDa5KwiPeARygHXKNMQHEy3rMopDR20XNZGbJzUtrwDC/KshlLDWyqdmhxZzCsdYmf2fWZPoxCEDyfIvdtNQH0PRkH6Q51g8rFO3Qzxh2LbItcDCOpmuOsV7ntNaERe3v/lP/zO8yn4N+yNPrekmPAAAAAElFTkSuQmCC Hello World base image

I think that this solution might be the simplest one at first. We can iterate on it but this will render nicely in list.standardnotes.org, be encrypted out of the box and does not require any changes to standard files server.

I can try making a PR on a test branch in https://github.com/sn-extensions/simple-markdown-editor (or minimal, or whatever markdown editor) to make drag'n'drop images convert to base64 (this mean file with mime-type of image/jp(e)g or image/png)

[dreamflasher]

This has been proposed and discussed in this thread (https://github.com/standardnotes/forum/issues/96#issuecomment-386204043) before. I am still in favor of this.

[RemiDesgrange]

Oh... sorry didn't see it, I read to quickly...

[dreamflasher]

Well I guess a PR would be very welcome, in the end nobody has to use the feature, if they don't like the downsides.

[JohJohOneThousand]

Here to add a little weight to the requests for inline/embedded images. It would be a killer feature and mean I could ditch Evernote. Thanks for the great work on a great tool.

[moughxyz]

Working on a prototype of this as we speak :) Now, it would only be attachments, like email attachments, and not inline. So you'd have to actually download them to preview them. Would that be sufficient for your use case?

[RemiDesgrange]

@mobitar Maybe make a PR as soon as possible, this way we could comment on the feature !

[moughxyz]

It would be a separate extension. Basically it will be in the same position as the Action Bar, but will display your attachments (encrypted) and allow you to drag and drop and download the attachments to decrypt them. The attachments would be stored encrypted in your own cloud provider (Dropbox and Google Drive initially, with more options to follow).

[JohJohOneThousand]

This will be a great improvement, thank you! Inline, one day :)

[technicalguy]

When this extension is complete will you add support to your Evernote ENEX → Standard Notes Tool for images? I'm trying to decide if I should hold out a bit longer so that I can migrate all in one go, or if I should just ditch evernote now and try to get my images/attachments later...

[moughxyz]

Because of the nature of the extension, it would be difficult to link the import process to the attachments extension. So I don't think this will be possible initially. You'd likely need to manually import attachments one by one. Hopefully as we expand the attachments product over time we can find solutions for incoming Evernote users.

[technicalguy]

Ah right, it's an extra layer of indirection. I also remembered that the mobile app does not support the extensions. Alas, after evaluating almost everything (except Keep and OneNote), I am still struggling to rid myself of Evernote. :'(

[kaushalmodi]

@mobitar Looking forward to inline or not image support. I want to port my Google Keep notes to here, but at the moment, I am losing all the media from there.

[moughxyz]

Encrypted attachments are now available through FileSafe: https://standardnotes.org/extensions/filesafe

[technicalguy]

@mobitar does FileSafe work on mobile?