forc index postgres create returns ReadFileError

[SilentCicero]

forc index postgres create example_database

📦 Downloading, unpacking, and bootstrapping database...

Error: error_type: ReadFileError
source: 
Some(Custom { kind: Other, error: InvalidArchive("Could not find central directory end") })
message: 
None

Caused by:
    invalid Zip archive

Note, this error also appeared when running:

forc index start \
    --embedded-database
    --fuel-node-host beta-3.fuel.network \
    --fuel-node-port 80 \
    --run-migrations

My forc and fuelup settings:

fuelup show

Default host: aarch64-unknown-linux-gnu
fuelup home: /home/ubuntu/.fuelup

installed toolchains
--------------------
latest-aarch64-unknown-linux-gnu (default)
nightly-aarch64-unknown-linux-gnu

active toolchain
-----------------
latest-aarch64-unknown-linux-gnu (default)
  forc : 0.42.1
    - forc-client
      - forc-deploy : 0.42.1
      - forc-run : 0.42.1
    - forc-doc : 0.42.1
    - forc-explore : 0.28.1
    - forc-fmt : 0.42.1
    - forc-index : 0.17.3
    - forc-lsp : 0.42.1
    - forc-tx : 0.42.1
    - forc-wallet : 0.2.3
  fuel-core : 0.18.3
  fuel-indexer : 0.17.3

fuels versions
---------------
forc : 0.43
forc-wallet : 0.43.0

[SilentCicero]

Likely related to this: https://github.com/faokunega/pg-embed/issues#issuecomment-1467907451

For any developer having this issue, you will have to go into the cache directory (where ever that is locally) and expand the ZIP files yourself. That should resolve the issue.

[ra0x3]

@SilentCicero

• I see you're in 0.17, i just tried with 0.18 ( I don't think there are any forc index postgres between versions though)
• You're creating your DB properly, but it's ephemeral by default, you need to persist it to disk forc index postgres create example_database --persistent
• Trying the Quickstart command I'm seeing that it's working?

 forc index start --embedded-database --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --run-migrations
✅ Successfully started the indexer service at PID 37938
▸▹▹▹▹ ⏱  Setting up database...
▪▪▪▪▪ ⏱  Setting up database...
                                                                                                                                       2023-07-11T14:06:40.982045Z  INFO fuel_indexer::commands::run: 65: Configuration: IndexerConfig { indexer_handler_timeout: 5, log_level: "info", verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelClientConfig { host: "beta-3.fuel.network", port: "80" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "5432", database: "postgres", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None }, rate_limit: RateLimitConfig { enabled: false, request_count: None, window_size: None }, replace_indexer: false }
2023-07-11T14:06:41.017363Z  INFO sqlx::postgres::notice: 157: identifier "graph_registry_graph_root_version_schema_name_schema_identifier_key" will be truncated to "graph_registry_graph_root_version_schema_name_schema_identifier"
2023-07-11T14:06:41.037789Z  INFO fuel_indexer_lib::utils: 132: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'

[ra0x3]

• Gonna close this.
• Be sure to use --persistent with forc index postgres create in order to persist the DB to disk (else it will be ephemeral). Further, passing --start to forc index postgres as well via forc index postgres --persistent --start ensures the DB is started.

[dmihal]

Hey, I'm reopening this, as I'm trying to do the quickstart with "--embedded-database" and getting the "Could not find central directory end" error

[ra0x3]

Thanks for the ping @dmihal

• The use of --embedded-database is very flaky.
• So much so, that we've added this big flashing sign (a while ago) when starting your service with that option

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Warning: Using the --embedded-database flag is experimental and should only be used for local development.

If the --embedded-database flag demonstrates flaky behavior on your machine, or doesn't work altogether, we recommend that you simply use the provided docker compose file here:
   https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/docker-compose.yaml.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

• This Could not find central directory end error is actually from pgembed and (as I've seen it) can imply that your folder structure is not as pgembed expects (for whatever reason)
• We highly recommend using Postgres through either Docker or homebrew
• We can leave this issue open, but it might take a while to resolve