nostr-protocol / nips

Nostr Implementation Possibilities
2.41k stars 586 forks source link

Enhance NIP-01 with IPFS CIDs for Decentralized Media Storage #1285

Closed besoeasy closed 3 months ago

besoeasy commented 5 months ago

Here is the current NIP1 Object

{ "id": "3237fd9a04af4cfcca803cc1825173c44f3ab360439820d30826d2016dfc0783", "pubkey": "f680ddd8c13f1ceab92edd3495f0fce747bb80c270e081f80c7401befb62da7c", "created_at": 1717795867, "kind": 1, "tags": [ [ "r", "https://i.ibb.co/JjYZQVN/graph.png" ] ], "content": "Bitcoin Looking Beautiful 😍 \n\nhttps://i.ibb.co/JjYZQVN/graph.png", "sig": "12b47bdf2db31b2630d219cc450b78a2a17a89f209b5fb16c7032c5264735052a3d8cabe1f60ae97957e2c581e07e6808eaa1b391d941a0915d9759825c52604" }

All we need to do is to add a "cid" field that resolves to IPFS Hash

vitorpamplona commented 5 months ago

I don't understand what you are trying to propose...

No client has an IPFS node client to pull images from the IPFS network. Are you proposing this becomes a requirement for all clients?

besoeasy commented 5 months ago

Description

Currently, NIP-01 supports decentralized storage for text-based content. However, it would be beneficial to extend this functionality to include multimedia files like images and videos. This can be achieved by utilizing InterPlanetary File System (IPFS) Content Identifiers (CIDs) to store and reference these files.

Example

  1. NIP-01 Update: Modify NIP-01 to include a new field for storing IPFS CIDs for multimedia files. This would allow users to upload and reference images and videos directly from their Nostr profiles.
  2. IPFS Integration: Integrate IPFS into the Nostr protocol to enable decentralized storage and retrieval of multimedia files. This would ensure that files are stored and accessed in a decentralized manner, reducing reliance on centralized servers.

Benefits

Steps to Implement

  1. Update NIP-01: Modify NIP-01 to include the new field for storing IPFS CIDs.
  2. Integrate IPFS: Integrate IPFS into the Nostr protocol to enable decentralized storage and retrieval of multimedia files.
vitorpamplona commented 5 months ago

Why not just place it as a tag?

besoeasy commented 5 months ago

Why not just place it as a tag?

There are several reasons why using IPFS CIDs as a native reference for multimedia content is more beneficial than simply placing it as a tag:

1. Native Support in Browsers and OS

IPFS is natively supported in some mainstream browsers like Brave and many Linux-based operating systems. This means that users can directly access IPFS content without needing additional software or plugins. By using IPFS CIDs, we can leverage this native support to provide a seamless user experience.

2. Decentralized Caching

One of the significant advantages of IPFS is that the entire network caches the content. This means that even if a specific gateway goes down, users can switch to another gateway and still access the content. In contrast, if a centralized server like nostr.build goes down, a significant portion of content images would become unavailable.

3. Flexibility and Portability

IPFS CIDs can be accessed through various gateways if native support is missing. This ensures that users can always access the content, even if their browser or OS does not natively support IPFS. The use of IPFS CIDs provides a flexible and portable solution for decentralized media storage.

4. Standardization

IPFS CIDs follow a standardized format, making it easier for different platforms and applications to integrate and support them. This standardization also enables better interoperability across different systems.

5. Security and Integrity

IPFS CIDs are based on the cryptographic hash of the content, ensuring that any changes to the content result in a different CID. This provides a robust mechanism for verifying the integrity and authenticity of the content.

In summary, using IPFS CIDs as a native reference for multimedia content offers several benefits, including native support in browsers and OS, decentralized caching, flexibility, standardization, and security. These advantages make IPFS CIDs a more robust and reliable solution compared to simply placing them as tags.

I been a experienced IPFS developer, i can show a small benchmark if the community wants and really proove that IPFS as media CDN would solve our dependencies on Centerlised Image Hosters

besoeasy commented 5 months ago

I forgot to add that IPFS, supports, all content types

From Images to direct .torrent files

Everything described in all of our NIPs is supported by IPFS

vitorpamplona commented 5 months ago

Why are you posting things from ChatGPT?

We know what IPFS is. That's not the question. The question is how are you going to integrate it. Changing NIP-01 is not needed to support IPFS.

In fact, you can integrate into your Nostr app without having to change any NIP. :)

alexgleason commented 5 months ago

The way to integrate IPFS with Nostr is easy. Just include a URL like https://<gateway>/ipfs/<cid>

This is backwards and forwards compatible. I've been doing it on Ditto: https://gitlab.com/soapbox-pub/ditto/-/blob/main/src/uploaders/IPFSUploader.ts?ref_type=heads

besoeasy commented 5 months ago

i am not using chatgpt, i am using a translator AI, i do not speak english

We know what IPFS is. That's not the question. The question is how are you going to integrate it. Changing NIP-01 is not needed to support IPFS.

i understand your question, i will post comparison, as soon i get some time, showing problems with NIP 1 Object

i will mention you here when i am done !

besoeasy commented 5 months ago

The way to integrate IPFS with Nostr is easy. Just include a URL like https://<gateway>/ipfs/<cid>

This is backwards and forwards compatible. I've been doing it on Ditto: https://gitlab.com/soapbox-pub/ditto/-/blob/main/src/uploaders/IPFSUploader.ts?ref_type=heads

we need to implement it as uri ipfs:// or "cid" field

to get rid of dependencies on gateways, if ipfs content is hardcoded with gateways, it makes it no difference then nostr.build or imgbb

alexgleason commented 5 months ago

The IPFS path gateway is standardized: https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-types

This is superior to ipfs:// URIs because it works on the legacy web, AND ipfs enabled tools can resolve it.

It is the same as an ipfs URI except it contains extra information. There is no reason not to use this.

besoeasy commented 5 months ago

The IPFS path gateway is standardized: https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-types

This is superior to ipfs:// URIs because it works on the legacy web, AND ipfs enabled tools can resolve it.

It is the same as an ipfs URI except it contains extra information. There is no reason not to use this.

example :

ipfs://QmRhZLf2ccHag8QLNEf8Jws4KXbtA4bx5K6xLf4WLa4kdn

and

https://bafybeibr5yx2gie53vmsgptybfpxfc6ykf5polikw3azhctygs2vfszcre.ipfs.dweb.link/

This demonstrates the versatility of IPFS in serving content through different methods.

My long-term vision for NOSTR is to integrate IPFS nodes as relay pools, similar to how we currently use Nostr relays. This would allow node runners to specify how much NOSTR and IPFS data they want to cache. While this is not a necessity at the moment, it is an improvement that will benefit NOSTR in the long term.

imo this makes NOSTR publicly more appealing

NOSTR is still new we can implement this, but in 2-3 years from now, this integration will be hard

AS NOSTR Community our priority is to keep protocol simple but at the same time, we are competing with a centralised Multi Billion Gaint, it doesn't look good with US telling our new users that images are not natively integrated thru a decenterlised protocol

alexgleason commented 5 months ago

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

besoeasy commented 5 months ago

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

doesn't sound decentralised, and requires always on server i assume

vitorpamplona commented 5 months ago

The http url renders great on Amethyst. :)

Screenshot 2024-06-07 at 6 28 18 PM

besoeasy commented 5 months ago

The http url renders great on Amethyst. :)

Screenshot 2024-06-07 at 6 28 18 PM

Exactly, it renders good as long as gateway is up

see this image

gateway goes up and down, and a good gateway for u, will not be best option for me

So if i am dictator in a country XYZ, all i need to do is to blacklist nost.build (few others CDN) and IPFS gateways and boom all of the country would not be able to access the content without VPN

vitorpamplona commented 5 months ago

Is there an Android app that serves IPFS images as a full node? If so, then we could just create a setting to load the image from the local app node instead of coding everything in Nostr.

besoeasy commented 5 months ago

Is there an Android app that serves IPFS images as a full node? If so, then we could just create a setting to load the image from the local app node instead of coding everything in Nostr.

thats smart, i really like this approach

I am afraid this can only done for non android for now, but it is possible, IPFS does have pure JS implementation

besoeasy commented 5 months ago

I will start a pilot with ipfs:// uri without changing NIP 01 as suggest by @vitorpamplona

i will talk to few clients devs to see if they can support this and see how it works, but i am afraid in order for this to work, we need to adopt this together

but something tells me, its not going to work well in full decentralised way unless we natively use IPFS and let them handle our troubles with files management & p2p delivery

vitorpamplona commented 5 months ago

Just develop a full IPFS client for Android first. Until that exists and becomes extremely reliable, it's useless to debate how to integrate with Nostr.

melvincarvalho commented 5 months ago

No changes needed in NIP-01 for this. If you want to add an IPFS URI such as ifps:// you can shoe-horn it into a tag. If you want to reference a CID but not be tied to IPFS transport, and use some other discovery method, I'd suggest using urn:cid:<CID> which decouples the hash, the transport and the discovery. Of course, you'd then need to figure out how to do discovery, perhaps using relays and lookups, but that's another NIP.

hzrd149 commented 5 months ago

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

doesn't sound decentralised, and requires always on server i assume

Blossom is not a decentralized protocol, but it is the simplest implementation of content addressable files. If we want censorship resistant media to fight link rot we don't need decentralized or p2p systems, at least not at the base layer. we need universal file IDs

IPFS CIDs are decent for this but they support multiple hashing algorithms which makes them complicated. Torrent hashes are good because they use a single hashing algorithm but they require you to chunk the file, which is complicated. Blossom proposes using just the sha256 to address files. since its trivial to compute and is universally supported.

Once you have a single universal ID for files, then it becomes easy to ask multiple servers for the file. in the case of IPFS its the gateways job to "ask multiple servers" using the p2p network to find the file. for Blossom its the clients job. Both of these approaches have their pros and cons, but I believe blossom approach is simpler since it does not rely on a p2p network or DHT table but instead relies on nostr and user specified server lists see docs

Both Blossom and IPFS store the files hash in the url which is critical for tamper resistant media and finding the file when its been taken down. However, again here I think blossom is simpler since it only has one format for urls https://blossom.example.com/<sha256>.png compared to IPFS

The point of this long reply isn't to say that IPFS wont work or isn't a good system for censorship resistant media. but simply that we don't need a system as complicated or heavy as IPFS to get censorship resistant media, we just need universal file IDs and servers that serve them by those IDs

besoeasy commented 5 months ago

The point of this long reply isn't to say that IPFS wont work or isn't a good system for censorship resistant media. but simply that we don't need a system as complicated or heavy as IPFS to get censorship resistant media, we just need universal file IDs and servers that serve them by those IDs

this is what i am trying to commend, either IPFS or our own in house system

besoeasy commented 5 months ago

Maybe easier would be to introduce a new tag in https://github.com/nostr-protocol/nips?tab=readme-ov-file#standardized-tags

["ipfs", "img", "IPFS CID"] ["ipfs", "vid", "IPFS CID"] ["ipfs", "other", "IPFS CID"]

This way it doesn't modify NIP 1, and the same time have IPFS standard for anyone looking to implement it

i see it as win-win for both sides of discussions

alexgleason commented 5 months ago

You're overthinking this. Just upload to a public gateway like I said and use a regular URL.

fiatjaf commented 5 months ago

If IPFS worked as advised we wouldn't need Nostr at all, so we should either close this or close the entirety of Nostr.