[Question] Rebuild Index

[Vinfall]

Description

My server had a filesystem failure which results in index lost, but all other files are kept intact. Is it possible to rebuild index from existing repos & opengist.db?

Details

I have (slightly outdated) backups and after manually comparing & checking the files in ~/.local/share/opengist, I found that only a specific file in opengist.index is missing due to the fs failure. Since all other files are still in place, I was wondering if it's possible to rebuild the index from existing files.

A quick search shows that root.bolt seems to be a BlotDB, not sure what's the zap file though. From log it seems to be a snapshot but I have no idea how this go database works. It's still acceptable if manual tweaking is required, key/value db sounds easy.

File tree

``` . ├── custom ├── log │ └── opengist.log ├── opengist-secret.key ├── opengist.db ├── opengist.index │ ├── index_meta.json │ └── store │ ├── 000000000032.zap (missing!) │ └── root.bolt ├── repos │ └── redacted │ ├── hash1 │ └── hash2 ├── sessions │ └── session-encrypt.key ├── ssh │ ├── opengist-ed25519 │ └── opengist-ed25519.pub ├── symlinks │ ├── config.yml -> /home/redacted/projects/opengist/config.yml │ └── opengist -> /home/redacted/projects/opengist/opengist └── tmp └── repos ```

Error log

```sh $ opengist --config config.yml Using YAML config file: /home/redacted/projects/opengist/config.yml No environment variables config specified. 2024/11/26 10:35:40 unable to load snapshot, failed to load segment: error opening bolt segment: open /home/redacted/.local/share/opengist/opengist.index/store/000000000032.zap: no such file or directory, continuing 10:35:40 ERR internal/index/bleve.go:34 Failed to open index error="error parsing mapping JSON: unexpected end of JSON input\nmapping contents:\n" ```

[thomiceli]

You may stop the server, delete the opengist.index/ directory then start the server again. Head to the admin panel and hit Index all gists.

This is something we should document in the docs :)

[Vinfall]

Oh, I did not even think about deleting that folder, glad it worked.

A possible suggestion: when I rebuild the index via Index all gists on admin panel, it does not show the process and the toast message stays quite long, so I can't really tell if the indexing is finished.

I wait a few seconds and restart the service, it's obviously working, but it's still better to have an indicator somehow.