Add Action script to build and push docker image to registry

[jasonyang-ee]

Issue

Lack of Docker image registry for none-tech user to pull directly

Solution

Action (.github/workflows/docker-build.yml) to build and push to registry.

Use

Push tags that starts with v to trigger build and push action. Example: git tag v1.0 && git push origin v1.0

Dependency

Repo owner must add repository actions secrets to accept this pull request for Action script to run correctly.

Steps to Recreate

1. Obtain package access token from UserIcon / Settings / Developer Settings / Personal access tokens (classic)
2. Add repository actions secrets for TOKEN_GITHUB
3. [optional] Uncomment Docker Hub section and add Add repository actions secrets for USERNAME_DOCKERHUB and TOKEN_DOCKERHUB to push image to docker hub.

[neonwatty]

Great idea! I'll try to get it in this weekend!

[jasonyang-ee]

I am glade that you are positive on this PR, so I have taken this one step further to also default docker compose with using image on github package.

[jasonyang-ee]

Issue

1. Docker deployment will have database persistence issue.
2. If dbs folder gets wiped from docker volume mount, it will fail to init with create.py

Error logs:

docker exec meme python meme_search/utilities/create.py
STARTING: collect_img_paths
SUCCESS: collect_img_paths ran successfully - image paths loaded from '/home/data/input/'
STARTING: collecting currently indexed names
Traceback (most recent call last):
  File "/home/meme_search/utilities/status.py", line 11, in get_current_indexed_img_names
    cursor.execute(query)
sqlite3.OperationalError: no such table: chunks_reverse_lookup

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/meme_search/utilities/create.py", line 19, in <module>
    process()
  File "/home/meme_search/utilities/create.py", line 8, in process
    old_imgs_to_be_removed, new_imgs_to_be_indexed = get_input_directory_status(img_dir, sqlite_db_path)
  File "/home/meme_search/utilities/status.py", line 24, in get_input_directory_status
    current_indexed_names = get_current_indexed_img_names(sqlite_db_path)
  File "/home/meme_search/utilities/status.py", line 18, in get_current_indexed_img_names
    raise ValueError(f"FAILURE: get_current_indexed_img_names failed with exception {e}")
ValueError: FAILURE: get_current_indexed_img_names failed with exception no such table: chunks_reverse_lookup

Temp Solution

Copy the data folder from repo and skip mounting of dbs folder. This will still have db persisting issue. Will need to inspect create.py before user can volume mount dbs folder.

[jasonyang-ee]

Sorry for the long thread.

New change

On Docker build, copy data folder to /home/init, then dbs content will get copied to user volume mounted database folder to help with init process.

This is done by using dedicated entrypoint bash file to expend the init process.

[neonwatty]

@jasonyang-ee my apologies! this was merged with the new workflow passing, but then i had issues when pulling the image locally so reverted.

The new workflow seems to have completed succesffully, but I'm getting an empty manifest when trying to pull the image.

What am I missing?

Apologies - I haven't worked with github's image repository so this likely something missed on my end. As soon as this clears up I will re-merge.

[jasonyang-ee]

@neonwatty I see the build is success from your action as well. I believe it's package public/private setting issue. The default github package is private. Please go to UserIcon / Profile / Package / meme_search / Package settings / change visibility.

[neonwatty]

@jasonyang-ee all right! The public setting seems to work. Some comments for slight adjustments - basically to include the input test images in init along with the dbs, so that the entrypoint provides a complete set when an empty data volume is mounted.

However that being said - is the entrypoint / Dockerfile copy the right way to gracefully fail when no data is mounted via the compose file?

At some point users may have large amounts of images in their data directory, which will be copied into their container.

And if they then want to push a version of their image to a registry - the data goes with it.

I'll get on integrating in the build action in the next day or two.

[neonwatty]

Re-merge complete!