/api/v0/get can't save file and not equal cmd: ipfs get

openpokergame commented 5 months ago

Checklist

[X] This is a bug report, not a question. Ask questions on discuss.ipfs.tech.
[X] I have searched on the issue tracker for my bug.
[X] I am running the latest kubo version or have an issue updating.

Installation method

built from source

Version

Kubo version: 0.30.0-dev-a07852a3f
Repo version: 15
System version: amd64/linux
Golang version: go1.22.4

Config

{
  "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Methods": [
        "PUT",
        "POST"
      ],
      "Access-Control-Allow-Origin": [
        "http://localhost:3000",
        "http://127.0.0.1:5001",
        "https://webui.ipfs.io"
      ]
    }
  },
  "Addresses": {
    "API": "/ip4/0.0.0.0/tcp/5001",
    "Announce": [],
    "AppendAnnounce": [],
    "Gateway": "/ip4/0.0.0.0/tcp/8080",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic-v1",
      "/ip4/0.0.0.0/udp/4001/quic-v1/webtransport",
      "/ip6/::/udp/4001/quic-v1",
      "/ip6/::/udp/4001/quic-v1/webtransport"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true
    }
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "Libp2pStreamMounting": false,
    "OptimisticProvide": false,
    "OptimisticProvideJobsPoolSize": 0,
    "P2pHttpProxy": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "DeserializedResponses": null,
    "DisableHTMLErrors": null,
    "ExposeRoutingAPI": null,
    "HTTPHeaders": {},
    "NoDNSLink": false,
    "NoFetch": false,
    "PublicGateways": null,
    "RootRedirect": ""
  },
  "Identity": {
    "PeerID": "12D3KooW9suBDD9ssMSGNGNtpXyWjRsxTYZnt1mvy1Bm9VCEx14t"
  },
  "Import": {
    "CidVersion": null,
    "HashFunction": null,
    "UnixFSChunker": null,
    "UnixFSRawLeaves": null
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {},
  "Routing": {
    "Methods": null,
    "Routers": null
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {},
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "RelayClient": {},
    "RelayService": {},
    "ResourceMgr": {},
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

when I run the cmd: curl -X POST "http://127.0.0.1:5001/api/v0/get?arg=/ipfs/QmQWM1qDPasxm5sXAQeVMfmhnECBzyYkLgfK23yPif1Ftx&output=/root/dev/kubo/cmd/ipfs/filem.mp4&progress=true" > geturl.mp4 the output can't save file, but /root/dev/kubo/cmd/ipfs/ is exist, why? only save geturl.mp4, when I run the cmd: ipfs get /ipfs/QmQWM1qDPasxm5sXAQeVMfmhnECBzyYkLgfK23yPif1Ftx -o /root/dev/kubo/cmd/ipfs/filem.mp4 , it is can be save so: why api result geturl.mp4 != filem.mp4, Why the sizes are different? 20240613201737

bitbytedance commented 5 months ago

curl -X POST "http://127.0.0.1:5001/api/v0/get?arg=/ipfs/QmQWM1qDPasxm5sXAQeVMfmhnECBzyYkLgfK23yPif1Ftx&output=/root/dev/kubo/cmd/ipfs/filem.mp4&progress=true" can't save file to /root/dev/kubo/cmd/ipfs/filem.mp4, why?

aschmahmann commented 5 months ago

curl can't write to your local machine and the HTTP API for the daemon doesn't directly do the writing. Instead what happens when you do ipfs get is it runs a kubo RPC client that:

Talks to the daemon (could be on your local machine or elsewhere if you wanted)
Fetches the data (e.g. might be a TAR if you're asking for a directory, could include progress info, etc.)
Writes the output where you told it to

So the output and progress are likely not things you want if you're just trying to save the file locally since they will . If you want fancier processing you can look at the RPC API clients in Go, JS, etc.

The likely easiest way for you to do fetching of a video locally via an HTTP API is to use the gateway API (e.g. curl http://127.0.0.1:8080/ipfs/<cid> > myfile.mp4).

bitbytedance commented 5 months ago

how can I download file via the rpc api?

lidel commented 5 months ago

@bitbytedance for trustless or deserialized retrieval over HTTP, you should use HTTP Gateway on port 8080 (see https://docs.ipfs.tech/reference/http/gateway/). Gateway, unlike RPC, uses correct HTTP GET method and you benefit from HTTP Caching and resuming via HTTP Range requests.

If you are ok with not benefiting from HTTP Caching, and for some reason really, really need to use POST-based RPC for fetching a single file, you can use /api/v0/cat with progress=false.

For future support questions like this, please use https://discuss.ipfs.tech/c/help/13 community forum.

We try to keep github only for bugs and feature requests, thanks!.

ipfs / kubo