search

simjanos-dev / LinguaCafe

LinguaCafe is a self-hosted software that helps language learners read foreign languages.
https://simjanos-dev.github.io/LinguaCafeHome/
GNU General Public License v3.0
812 stars 25 forks source link

Docker deployment issues #59

Closed sergiolaverde0 closed 5 months ago

sergiolaverde0 commented 6 months ago

This issue is here to keep track of all problems regarding building the images and running the contsiners. Issues regarding features, dependencies, networking and other should not be reported here.

Issues that happen only in the dev branch should mention it explicitly.

simjanos-dev commented 6 months ago

mac

Someone else also had database healthcheck issues. He uses mac. It was after first install, and docker compose down docker compose up -d fixed it.

He had to add platform: linux/x86_64 code in the docker compose file for it to work:

    mysql:
        platform: linux/x86_64
        image: mysql:5.7.22
    python:
        platform: linux/x86_64
        container_name: linguacafe-python-service

I think the second problem will be fixed if the image will be built on GitHub.

sergiolaverde0 commented 6 months ago

I have been shutting on and off the service for some minutes trying to reproduce with no luck. We might have to add an option to have MySQL logs since that seems to be the root cause of many issues and debugging is hard.

As for the second one, I'm fairly certain that was just a case of "turn it off and then back on" solving it, and the extra lines in the compose file are a red herring. Just to be sure, where they running a Mac with Apple silicon?

simjanos-dev commented 6 months ago

Sorry, I forgot to add more information about the second issue. He got this error message when tried to start the docker image the first time:

no matching manifest for linux/arm64/v8 in the manifest list entries.

After adding the platform line(I found it in an answer while googling), It started building the image, but the Python failed. So he also added the line to python. After that it built the image, but failed with the health check. I think they were running it on Apple silicon, but not sure.

Is it possible, that the health check issue is not an actual issue, it's just that for some people MySQL takes longer time to start up? I am building a fresh image now, if it fails health check, I'll comment the log here.

simjanos-dev commented 6 months ago

Health check issue happened again. It also freeze my PC, but that part is because it's slow.

Here is the complete logs from the database container:

2024-01-14 16:37:36 Initializing database
2024-01-14 16:37:36 2024-01-14T15:37:36.308534Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2024-01-14 16:38:07 2024-01-14T15:38:07.237864Z 0 [Warning] InnoDB: New log files created, LSN=45790
2024-01-14 16:38:08 2024-01-14T15:38:08.165551Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2024-01-14 16:38:08 2024-01-14T15:38:08.318596Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: ea9f0bcb-b2f2-11ee-aba0-0242ac120003.
2024-01-14 16:38:08 2024-01-14T15:38:08.326524Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2024-01-14 16:38:08 2024-01-14T15:38:08.337726Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2024-01-14 16:38:14 2024-01-14T15:38:14.216706Z 1 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.216866Z 1 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.217040Z 1 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.217473Z 1 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.217551Z 1 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.217620Z 1 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.218097Z 1 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:14 2024-01-14T15:38:14.218186Z 1 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:44 Database initialized
2024-01-14 16:38:44 Initializing certificates
2024-01-14 16:38:44 Generating a 2048 bit RSA private key
2024-01-14 16:38:44 .................+++
2024-01-14 16:38:44 ..........................................+++
2024-01-14 16:38:44 unable to write 'random state'
2024-01-14 16:38:44 writing new private key to 'ca-key.pem'
2024-01-14 16:38:44 -----
2024-01-14 16:38:44 Generating a 2048 bit RSA private key
2024-01-14 16:38:44 ......+++
2024-01-14 16:38:44 ..+++
2024-01-14 16:38:44 unable to write 'random state'
2024-01-14 16:38:44 writing new private key to 'server-key.pem'
2024-01-14 16:38:44 -----
2024-01-14 16:38:45 Generating a 2048 bit RSA private key
2024-01-14 16:38:45 .......+++
2024-01-14 16:38:45 .......+++
2024-01-14 16:38:45 unable to write 'random state'
2024-01-14 16:38:45 writing new private key to 'client-key.pem'
2024-01-14 16:38:45 -----
2024-01-14 16:38:45 Certificates initialized
2024-01-14 16:38:46 MySQL init process in progress...
2024-01-14 16:38:46 2024-01-14T15:38:46.892119Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2024-01-14 16:38:46 2024-01-14T15:38:46.897394Z 0 [Note] mysqld (mysqld 5.7.22) starting as process 129 ...
2024-01-14 16:38:46 2024-01-14T15:38:46.904569Z 0 [Note] InnoDB: PUNCH HOLE support available
2024-01-14 16:38:46 2024-01-14T15:38:46.904631Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2024-01-14 16:38:46 2024-01-14T15:38:46.904751Z 0 [Note] InnoDB: Uses event mutexes
2024-01-14 16:38:46 2024-01-14T15:38:46.904780Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2024-01-14 16:38:46 2024-01-14T15:38:46.904795Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2024-01-14 16:38:46 2024-01-14T15:38:46.904816Z 0 [Note] InnoDB: Using Linux native AIO
2024-01-14 16:38:46 2024-01-14T15:38:46.905297Z 0 [Note] InnoDB: Number of pools: 1
2024-01-14 16:38:46 2024-01-14T15:38:46.906214Z 0 [Note] InnoDB: Using CPU crc32 instructions
2024-01-14 16:38:46 2024-01-14T15:38:46.908764Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2024-01-14 16:38:46 2024-01-14T15:38:46.931601Z 0 [Note] InnoDB: Completed initialization of buffer pool
2024-01-14 16:38:46 2024-01-14T15:38:46.937908Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2024-01-14 16:38:47 2024-01-14T15:38:47.109801Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2024-01-14 16:38:47 MySQL init process in progress...
2024-01-14 16:38:47 2024-01-14T15:38:47.624827Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2024-01-14 16:38:47 2024-01-14T15:38:47.626232Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2024-01-14 16:38:47 2024-01-14T15:38:47.889020Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2024-01-14 16:38:47 2024-01-14T15:38:47.892911Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2024-01-14 16:38:47 2024-01-14T15:38:47.893107Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2024-01-14 16:38:47 2024-01-14T15:38:47.893693Z 0 [Note] InnoDB: Waiting for purge to start
2024-01-14 16:38:47 2024-01-14T15:38:47.980247Z 0 [Note] InnoDB: 5.7.22 started; log sequence number 2592582
2024-01-14 16:38:47 2024-01-14T15:38:47.981555Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2024-01-14 16:38:47 2024-01-14T15:38:47.982491Z 0 [Note] Plugin 'FEDERATED' is disabled.
2024-01-14 16:38:48 2024-01-14T15:38:48.075531Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2024-01-14 16:38:48 2024-01-14T15:38:48.140559Z 0 [Warning] CA certificate ca.pem is self signed.
2024-01-14 16:38:48 2024-01-14T15:38:48.151719Z 0 [Note] InnoDB: Buffer pool(s) load completed at 240114 15:38:48
2024-01-14 16:38:48 2024-01-14T15:38:48.675645Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2024-01-14 16:38:48 2024-01-14T15:38:48.706902Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.707538Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.707975Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.711688Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.711715Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.712098Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.728486Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.728516Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:48 2024-01-14T15:38:48.834364Z 0 [Note] Event Scheduler: Loaded 0 events
2024-01-14 16:38:48 2024-01-14T15:38:48.834576Z 0 [Note] mysqld: ready for connections.
2024-01-14 16:38:48 Version: '5.7.22'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server (GPL)
2024-01-14 16:38:56 Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
2024-01-14 16:38:56 Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
2024-01-14 16:38:59 Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
2024-01-14 16:38:59 Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2024-01-14 16:38:59 2024-01-14T15:38:59.898453Z 5 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898511Z 5 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898530Z 5 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898567Z 5 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898577Z 5 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898593Z 5 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898658Z 5 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.898677Z 5 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 mysql: [Warning] Using a password on the command line interface can be insecure.
2024-01-14 16:38:59 mysql: [Warning] Using a password on the command line interface can be insecure.
2024-01-14 16:38:59 mysql: [Warning] Using a password on the command line interface can be insecure.
2024-01-14 16:38:59 mysql: [Warning] Using a password on the command line interface can be insecure.
2024-01-14 16:38:59 2024-01-14T15:38:59.983013Z 9 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983047Z 9 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983055Z 9 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983114Z 9 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983119Z 9 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983159Z 9 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983739Z 9 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 2024-01-14T15:38:59.983753Z 9 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2024-01-14 16:38:59 
2024-01-14 16:38:59 2024-01-14T15:38:59.995077Z 0 [Note] Giving 0 client threads a chance to die gracefully
2024-01-14 16:38:59 2024-01-14T15:38:59.995105Z 0 [Note] Shutting down slave threads
2024-01-14 16:38:59 2024-01-14T15:38:59.995110Z 0 [Note] Forcefully disconnecting 0 remaining clients
2024-01-14 16:38:59 2024-01-14T15:38:59.995117Z 0 [Note] Event Scheduler: Purging the queue. 0 events
2024-01-14 16:38:59 2024-01-14T15:38:59.995999Z 0 [Note] Binlog end
2024-01-14 16:38:59 2024-01-14T15:38:59.998627Z 0 [Note] Shutting down plugin 'ngram'
2024-01-14 16:38:59 2024-01-14T15:38:59.998648Z 0 [Note] Shutting down plugin 'partition'
2024-01-14 16:38:59 2024-01-14T15:38:59.998653Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2024-01-14 16:38:59 2024-01-14T15:38:59.998657Z 0 [Note] Shutting down plugin 'ARCHIVE'
2024-01-14 16:38:59 2024-01-14T15:38:59.998660Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2024-01-14 16:38:59 2024-01-14T15:38:59.998691Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2024-01-14 16:38:59 2024-01-14T15:38:59.998695Z 0 [Note] Shutting down plugin 'MyISAM'
2024-01-14 16:38:59 2024-01-14T15:38:59.998708Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2024-01-14 16:38:59 2024-01-14T15:38:59.998713Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2024-01-14 16:38:59 2024-01-14T15:38:59.998716Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2024-01-14 16:38:59 2024-01-14T15:38:59.998719Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998721Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2024-01-14 16:38:59 2024-01-14T15:38:59.998725Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998728Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998731Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2024-01-14 16:38:59 2024-01-14T15:38:59.998733Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998736Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2024-01-14 16:38:59 2024-01-14T15:38:59.998739Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2024-01-14 16:38:59 2024-01-14T15:38:59.998742Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2024-01-14 16:38:59 2024-01-14T15:38:59.998746Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2024-01-14 16:38:59 2024-01-14T15:38:59.998748Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2024-01-14 16:38:59 2024-01-14T15:38:59.998751Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2024-01-14 16:38:59 2024-01-14T15:38:59.998754Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2024-01-14 16:38:59 2024-01-14T15:38:59.998757Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998761Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO'
2024-01-14 16:38:59 2024-01-14T15:38:59.998764Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998767Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2024-01-14 16:38:59 2024-01-14T15:38:59.998771Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2024-01-14 16:38:59 2024-01-14T15:38:59.998773Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2024-01-14 16:38:59 2024-01-14T15:38:59.998776Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2024-01-14 16:38:59 2024-01-14T15:38:59.998780Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2024-01-14 16:38:59 2024-01-14T15:38:59.998784Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2024-01-14 16:38:59 2024-01-14T15:38:59.998788Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2024-01-14 16:38:59 2024-01-14T15:38:59.998792Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2024-01-14 16:38:59 2024-01-14T15:38:59.998796Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998800Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2024-01-14 16:38:59 2024-01-14T15:38:59.998805Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2024-01-14 16:38:59 2024-01-14T15:38:59.998809Z 0 [Note] Shutting down plugin 'InnoDB'
2024-01-14 16:38:59 2024-01-14T15:38:59.998900Z 0 [Note] InnoDB: FTS optimize thread exiting.
2024-01-14 16:38:59 2024-01-14T15:38:59.999122Z 0 [Note] InnoDB: Starting shutdown...
2024-01-14 16:39:00 2024-01-14T15:39:00.099598Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2024-01-14 16:39:00 2024-01-14T15:39:00.100470Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 240114 15:39:00
2024-01-14 16:40:10 2024-01-14T15:40:10.042753Z 0 [Note] InnoDB: Shutdown completed; log sequence number 12363164
2024-01-14 16:40:10 2024-01-14T15:40:10.045710Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2024-01-14 16:40:10 2024-01-14T15:40:10.045752Z 0 [Note] Shutting down plugin 'MEMORY'
2024-01-14 16:40:10 2024-01-14T15:40:10.045762Z 0 [Note] Shutting down plugin 'CSV'
2024-01-14 16:40:10 2024-01-14T15:40:10.045768Z 0 [Note] Shutting down plugin 'sha256_password'
2024-01-14 16:40:10 2024-01-14T15:40:10.045772Z 0 [Note] Shutting down plugin 'mysql_native_password'
2024-01-14 16:40:10 2024-01-14T15:40:10.046199Z 0 [Note] Shutting down plugin 'binlog'
2024-01-14 16:40:10 2024-01-14T15:40:10.057048Z 0 [Note] mysqld: Shutdown complete
2024-01-14 16:40:10 
2024-01-14 16:40:10 
2024-01-14 16:40:10 MySQL init process done. Ready for start up.
2024-01-14 16:40:10 
2024-01-14 16:40:10 2024-01-14T15:40:10.453593Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2024-01-14 16:40:10 2024-01-14T15:40:10.470614Z 0 [Note] mysqld (mysqld 5.7.22) starting as process 1 ...
2024-01-14 16:40:10 2024-01-14T15:40:10.486672Z 0 [Note] InnoDB: PUNCH HOLE support available
2024-01-14 16:40:10 2024-01-14T15:40:10.486853Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2024-01-14 16:40:10 2024-01-14T15:40:10.486949Z 0 [Note] InnoDB: Uses event mutexes
2024-01-14 16:40:10 2024-01-14T15:40:10.487053Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2024-01-14 16:40:10 2024-01-14T15:40:10.487150Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2024-01-14 16:40:10 2024-01-14T15:40:10.487241Z 0 [Note] InnoDB: Using Linux native AIO
2024-01-14 16:40:10 2024-01-14T15:40:10.487766Z 0 [Note] InnoDB: Number of pools: 1
2024-01-14 16:40:10 2024-01-14T15:40:10.488945Z 0 [Note] InnoDB: Using CPU crc32 instructions
2024-01-14 16:40:10 2024-01-14T15:40:10.495125Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2024-01-14 16:40:10 2024-01-14T15:40:10.511636Z 0 [Note] InnoDB: Completed initialization of buffer pool
2024-01-14 16:40:10 2024-01-14T15:40:10.516326Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2024-01-14 16:40:10 2024-01-14T15:40:10.538645Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2024-01-14 16:40:10 2024-01-14T15:40:10.601812Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2024-01-14 16:40:10 2024-01-14T15:40:10.602458Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2024-01-14 16:40:11 2024-01-14T15:40:11.009240Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2024-01-14 16:40:11 2024-01-14T15:40:11.020524Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2024-01-14 16:40:11 2024-01-14T15:40:11.020589Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2024-01-14 16:40:11 2024-01-14T15:40:11.022087Z 0 [Note] InnoDB: Waiting for purge to start
Error grabbing logs: invalid character '\x00' looking for beginning of value
sergiolaverde0 commented 6 months ago

Got it.

Older versions of MySQL don't have ARM images in Docker Hub so that's why the initial pull failed after docker successfully identified the platform. Moving to MySQL 8 will introduce support for it, and the project images will be built by commenting out the lines at PR #50 to enable at least arm64v8. It is possible to enable other platforms, but since MySQL does not have docker support for them it would be pointless.

We probably should add the supported platforms in the README.md.

simjanos-dev commented 6 months ago

What should I write exactly in README.md? Windows, Linux and how should I phrase Mac, arm64v8 and older?

sergiolaverde0 commented 6 months ago

For MySQL try changing interval and timeout to 15s and retries to 8 in docker-compose.yml. If that solves it, then my health check was indeed too aggressive and will have to be fixed.

For platforms, they mean the hardware and not the OS, so it would be x64 (most desktop computers from the past decade) and Armv8 (Raspberry Pi 3 and newer and Apple silicon). Raspberry Pis must be running a x64 OS otherwise they are taken as Armv7.

simjanos-dev commented 6 months ago

Okay. I'll add Supported hardware: X64 and Armv8 to the readme.

I'll change the timeout too, and will probably release the next update tomorrow with the image built on github if everything works.

How should the update process be done for people who have already started using it, and how should it look like after v0004? (I probably can figure it out, but do not know best practices, and do not want to mess it up again)

Thank you so much for helping!

sergiolaverde0 commented 6 months ago

Update will be... interesting. A seamless upgrade can be achieved by mounting the current folders as volumes, so that path/to/this/repo/storage/app/dictionaries is the dict folder, path/to/this/repo/storage/logs is the logs folder and /path/to/this/repo/docker/mysql is the database folder.

I have opened PR #64 to include this and other things into the README.md and other changes discussed here.

simjanos-dev commented 6 months ago

All right. This is so close. Everything works, except when I first started up first from a fresh install with the github docker image, the database didn't have users table, so the migrations didn't run. I've run docker compose down and docker compose up -d, and it was working. 

[+] Running 72/3
 ✔ mysql 11 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                                              662.5s 
 ✔ python 26 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                              425.1s 
 ✔ webserver 32 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                     713.5s 
[+] Running 4/4
 ✔ Network linguacafe_linguacafe        Created                                                                                                        0.1s 
 ✔ Container linguacafe-database        Healthy                                                                                                        1.5s 
 ✔ Container linguacafe-python-service  Started                                                                                                        1.5s 
 ✔ Container linguacafe-webserver       Started                                                                                                        0.9s 
laptop@laptop-mrcwx0:~/Projects/LinguaCafeGithubTest$  docker compose down
[+] Running 4/3
 ✔ Container linguacafe-python-service  Removed                                                                                                        1.8s 
 ✔ Container linguacafe-webserver       Removed                                                                                                        1.3s 
 ✔ Container linguacafe-database        Removed                                                                                                        1.4s 
 ✔ Network linguacafe_linguacafe        Removed                                                                                                        0.1s 
laptop@laptop-mrcwx0:~/Projects/LinguaCafeGithubTest$ docker compose up -d
[+] Running 4/4
 ✔ Network linguacafe_linguacafe        Created                                                                                                        0.1s 
 ✔ Container linguacafe-python-service  Started                                                                                                        0.1s 
 ✔ Container linguacafe-database        Healthy                                                                                                        0.1s 
 ✔ Container linguacafe-webserver       Started                                                                                                        0.1s

docker-compose.yml file has this line, which means it should create this table on first start up: MYSQL_DATABASE: linguacafe

Any idea?

sergiolaverde0 commented 6 months ago

No clue, I have created a stack from scratch and new volumes and it seemed to work fine but I didn't go too deep. This here is my log:

2024-01-14 22:23:02+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.35-1.el8 started.
2024-01-14 22:23:02+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-01-14 22:23:02+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.35-1.el8 started.
2024-01-14 22:23:02+00:00 [Note] [Entrypoint]: Initializing database files
2024-01-14T22:23:02.919853Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2024-01-14T22:23:02.919941Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.35) initializing of server in progress as process 81
2024-01-14T22:23:02.926329Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-01-14T22:23:03.286192Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2024-01-14T22:23:04.276867Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2024-01-14 22:23:09+00:00 [Note] [Entrypoint]: Database files initialized
2024-01-14 22:23:09+00:00 [Note] [Entrypoint]: Starting temporary server
mysqld will log errors to /var/lib/mysql/aafb49119736.err
mysqld is running as pid 127
2024-01-14 22:23:10+00:00 [Note] [Entrypoint]: Temporary server started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2024-01-14 22:23:12+00:00 [Note] [Entrypoint]: Creating database linguacafe
2024-01-14 22:23:12+00:00 [Note] [Entrypoint]: Creating user linguacafe
2024-01-14 22:23:12+00:00 [Note] [Entrypoint]: Giving user linguacafe access to schema linguacafe

2024-01-14 22:23:12+00:00 [Note] [Entrypoint]: Stopping temporary server
2024-01-14 22:23:16+00:00 [Note] [Entrypoint]: Temporary server stopped

2024-01-14 22:23:16+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.

2024-01-14T22:23:16.263316Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2024-01-14T22:23:16.264137Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.35) starting as process 1
2024-01-14T22:23:16.269845Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-01-14T22:23:16.346731Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2024-01-14T22:23:16.495089Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2024-01-14T22:23:16.495117Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2024-01-14T22:23:16.496222Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2024-01-14T22:23:16.508072Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2024-01-14T22:23:16.508104Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.35'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

Could you give me yours to compare if there are any differences? This is my compose file for the test:

version: '3'
name: linguacafe

networks:
    linguacafe:
        driver: bridge

services:
    webserver:
        image: ghcr.io/simjanos-dev/linguacafe-webserver:dev
        container_name: linguacafe-webserver
        depends_on:
            mysql:
                condition: service_healthy
        volumes:
            - ~/TMP/volume/dicts:/var/www/html/storage/app/dictionaries
            - ~/TMP/volume/logs:/var/www/html/storage/logs
        ports:
            - "9191:80"
            - "3000:3000"
        networks:
            - linguacafe
        extra_hosts:
            - "host.docker.internal:host-gateway"
    mysql:
        image: mysql:8.0
        container_name: linguacafe-database
        restart: unless-stopped
        tty: true
        ports:
          - "3308:3306"
        volumes:
          - ~/TMP/volume/mysql:/var/lib/mysql
        environment:
          MYSQL_DATABASE: linguacafe
          MYSQL_USER: linguacafe
          MYSQL_PASSWORD: linguacafe
          MYSQL_ROOT_PASSWORD: linguacafe
          SERVICE_NAME: linguacafe
        healthcheck:
            test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
            interval: 15s
            timeout: 15s
            retries: 8
        command: mysqld --general-log=1 --general-log-file=/var/lib/mysql/general-log.log
        networks:
            - linguacafe
    python:
        container_name: linguacafe-python-service
        command: /app/manage.py runserver 0.0.0.0:8678
        restart: unless-stopped
        tty: true
        image: ghcr.io/simjanos-dev/linguacafe-python-service:dev
        volumes:
            - ~/TMP/volume/dicts:/var/www/html/storage/app/dictionaries
        networks:
            - linguacafe
simjanos-dev commented 6 months ago 
version: '3'
name: linguacafe

networks:
    linguacafe:
        driver: bridge

services:
    webserver:
        container_name: linguacafe-webserver
        image: ghcr.io/simjanos-dev/linguacafe-webserver:dev
        depends_on:
            mysql:
                condition: service_healthy
        volumes:
            - ./dictionaries:/var/www/html/storage/app/dictionaries
            - ./logs:/var/www/html/storage/logs
        ports:
            - "9191:80"
            - "3000:3000"
        networks:
            - linguacafe
        extra_hosts:
            - "host.docker.internal:host-gateway"
    mysql:
        image: mysql:8.0
        container_name: linguacafe-database
        restart: unless-stopped
        tty: true
        ports:
          - "3308:3306"
        volumes:
          - ./database:/var/lib/mysql
        environment:
          MYSQL_DATABASE: linguacafe
          MYSQL_USER: linguacafe
          MYSQL_PASSWORD: linguacafe
          MYSQL_ROOT_PASSWORD: linguacafe
          SERVICE_NAME: linguacafe
        healthcheck:
            test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
            interval: 15s
            timeout: 15s
            retries: 8
        command: mysqld --general-log=1 --general-log-file=/var/lib/mysql/general-log.log
        networks:
            - linguacafe
    python:
        container_name: linguacafe-python-service
        command: /app/manage.py runserver 0.0.0.0:8678
        restart: unless-stopped
        tty: true
        image: ghcr.io/simjanos-dev/linguacafe-python-service:dev
        volumes:
            - ./dictionaries:/var/www/html/storage/app/dictionaries
        networks:
            - linguacafe

I've got the error when I opened up the browser. Mabe I was just impatient, and opened it up before the migrations run?

I don't know. If it works for you, it's probably a problem with me I'll release it tomorrow, and wait for feedback.

Thank you so much for the lot of help! It would have taken me weeks to learn all this and try it out. Now people have a simple install and update process.

sergiolaverde0 commented 6 months ago

I tried to replicate it by smashing F5 after running docker-compose up -d with no result. I'm fairly certain we have a bug here, just one that is hard to replicate. It will very likely bite us in the arse later but I agree on releasing it, no point chasing perfection.

sergiolaverde0 commented 6 months ago

I think the second problem will be fixed if the image will be built on GitHub.

Hello, I want to mention arm64 is still commented out in the dev branch so users running it in Apple silicon and Raspberry pi will have the same issue again unless fixed.

simjanos-dev commented 6 months ago

I have to add it below the linux/amd64, like this:

with:
          file: "./docker/PhpDockerfile"
          # extra platforms can be added here:
          # linux/amd64
          # linux/arm/v6
          # linux/arm/v7
          # linux/386
          platforms: |
            linux/amd64
            linux/arm64

and

with:
          file: "./docker/PythonDockerfile"
          platforms: |
            linux/amd64
            linux/arm64

right?

sergiolaverde0 commented 6 months ago

Yes, just like that.

simjanos-dev commented 6 months ago

I have added them, thank you!

I'm testing newly added dictionaries. I'll merge it into main sometime today, I'm not adding more things to v0.4.

Someone on discord also had crashes/freezes with an 8GB Linux machine after building the images. I didn't have any problems anymore when I downloaded the image from github, so I hope it helps with lower spec PCs.

simjanos-dev commented 6 months ago

Should I be worried about any problems with ghcr limits while testing? If my understanding is correct, then there is no limit for public packages. I think it will takes 2-3~ commits to dev before I push it to main, and it will build it every time I push a commit.

simjanos-dev commented 6 months ago

I'll test the current dev branch after the new image has been built. If it works I'll change the image building from dev to main branch, and will merge the new update in.

simjanos-dev commented 6 months ago

I've realized there is an issue with docker containers. LinguaCafe/storage/app/images/book_images/ is inside the container. If someone pulls the next version, all their book cover images will be lost. This also happened if someone updated from v0.3 -> v0.4 and kept their database. I could add a migration to set all of their images to the default images, so there wouldn't be a code error. But I'm not sure if there is a way to save their cover images without complicated update process.

sergiolaverde0 commented 6 months ago

In theory we could simply add yet another volume that would be mounted at your/linguacafe/covers/folder and binds to /var/www/html/storage/app/images/book_images. Unlike dictionaries, users are not expected to manually place content there so is fine if docker automatically creates the folder by passing it a valid location.

I eventually will go back to butting heads with the permissions on the log file so that users can simply mount a single storage folder and ease their deployment experience, but for now this should be a decent workaround.

simjanos-dev commented 6 months ago

What is the issue with the log file permissions?

I think this is starting to get too complicated for the users. They have to add writing permissions to temp folder at the first installment, and they have to create 5 folders, and if we mount the whole storage directory, they will have to move stuff around again. The mac user (it worked for them) didn't understand what to do with folders from the readme until i explained it to them in detail and examples.

I think I'll wait with the next update until it's figured out. (Was planning on making it a bigger one anyway)

Also, i think we should replace the default /your/path/to lines with paths pointing to the same directory(./temp, ./directories). It would be easier to deploy, and they still can modify these paths if they want to.

sergiolaverde0 commented 6 months ago

What is the issue with the log file permissions?

If the user doesn't create the logs folder beforehand, docker creates one but the Laravel process doesn't have writing permission to that location so it creates an error. You can replicate it by not creating the folder before running the containers and doing anything that triggers a write to log.

I think this is starting to get too complicated for the users.

Agreed.

They have to add writing permissions to temp folder at the first installment

They do? I expected that part to at least not be needed. Users running chmod is bothersome at best and dangerous at worst.

and they have to create 5 folders

This is the nasty part: actually only the logs and dictionaries must be created first because they need more lax permissions. But making them create two and then mount five in the containers is probably as confusing.

if we mount the whole storage directory, they will have to move stuff around again.

Yes, it would be a breaking change that needs manual intervention. But at least it would solve all persistence issues for the foreseeable future.

Also, i think we should replace the default /your/path/to lines with paths pointing to the same directory(./temp, ./directories). It would be easier to deploy, and they still can modify these paths if they want to.

Yes, a lot of similar projects have an example of how volumes are mounted to show people a more natural folder layout. They can do this right now if they want, but many probably don't know it.

I will be drafting something satisfactory enough to be merged, but might take a couple of days.

simjanos-dev commented 6 months ago

Thank you! I think the next patch will be out next week, or after. This time I want to add more things.

I will move the flags out of the storage/app/images directory, so if we mount the whole storage folder it won't be an issue.

I think we should only need 2 folders: storage and 'database'. What do you think about creating a different branch, where we only store the required empty directories, and the docker-compose.yml. This would allow the users to get the server up and running with a git clone and docker compose up -d command. The only thing I do not know yet is how could we make the permissions be set automatically, without requiring any user action.

sergiolaverde0 commented 6 months ago

I think git clone preserves permissions, so the branch is a galaxy brain move!

simjanos-dev commented 6 months ago

And I think we can link the docker-compose.yml file from the main branch, so the deployment (I'm not sure what to call it) branch won't even need manual updating.

simjanos-dev commented 6 months ago

I've looked it up, and I think git clone does not preserve permissions except the execution permission. But I think we could just do this instead:

git clone https://github.com/simjanos-dev/LinguaCafe.git && chmod 777 ./LinguaCafe -R && cd LinguaCafe && docker compose up -d

This way people would only have to run one single command. May have to add sudo to the chmod. What do you think?

sergiolaverde0 commented 6 months ago

I would rather not have to rely on chmod 777 and sudo is my last retort. I have devised a solution that relies on containers running as user 1000 in group 1000, which is probably a sane default. It at least seems to not throw immediate errors:

Screenshot_20240117_195823

I will be testing for correct and sane persistence, for which I need to know what is persistent at what could have escaped us. Right now I have in mind creating a user, adding a book with various chapters and a cover image, reading the first chapters and maybe highlighting a few ones. then restart the service and check that cover images are there, known words are still known and unknown words are still unknown. What else?

simjanos-dev commented 6 months ago

Importing dictionary. Both supported and custom .csv (you can just create one with a few rows). I don't think there is anything more to test related to files.

There are more and more concepts in the docker process that I do not understand enough to write it myself properly, so please make sure you test it to a level, where I can safely merge it in.

I do not mind it not being perfect, I wouldn't mind at all going with chmod 777, I just don't want anyone who uses LinguaCafe to have any problems. So far it feels like everything just got more and more problematic and complex.

But thank you for spending so much time making it better! I really appreciate it!

simjanos-dev commented 6 months ago

I found out that there are 11.723 svg files that area missing from the software, and needed for drawing Chinese characters on the Kanji page. The total size of them is 50MB.

Should I just upload them to the main GitHub repository, or should we put a command in the dockerfile to download them while building the image?

sergiolaverde0 commented 6 months ago

Assuming the source where they are hosted is reliable downloading them is the way to go, but naturally keep a copy handy to use as a backup.

TheNomad11 commented 6 months ago

I cannot get rid of the database error even after restarting a few times. I installled it with the provide compose file on an Debian VPS:

   Illuminate\Database\QueryException 

  SQLSTATE[HY000] [1045] Access denied for user 'linguacafe'@'192.168.96.4' (using password: YES) (SQL: select * from information_schema.tables where table_schema = linguacafe and table_name = migrations and table_type = 'BASE TABLE')

  at vendor/laravel/framework/src/Illuminate/Database/Connection.php:712
simjanos-dev commented 6 months ago

I think I had a different message. I did have access to it, I just didn't have the linguacafe table created.

If you run this command below while the server is running, do you get an error message, or does it let you in?

docker exec -ti linguacafe-database mysql -ulinguacafe -plinguacafe

If does let you in, does it fix the problem if you run this next command inside MySql?

create database linguacafe;

Edit: Also, could you please copy-paste here your edited docker-compose.yml file?

TheNomad11 commented 6 months ago

Thanks for your quick reply @simjanos-dev This command did not let me in:

~/linguacafe$ sudo docker exec -ti linguacafe-database mysql -ulinguacafe -plinguacafe
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'linguacafe'@'localhost' (using password: YES)

Here is the compose file. when I installed it first I changed the mysql passwords, then I changed it back to the defauilt one (see below) 

version: "3"
name: linguacafe

networks:
    linguacafe:
        driver: bridge

services:
    webserver:
        image: ghcr.io/simjanos-dev/linguacafe-webserver:main
        container_name: linguacafe-webserver
        depends_on:
            mysql:
                condition: service_healthy
        volumes:
            - ./dict:/var/www/html/storage/app/dictionaries
            - ./logs:/var/www/html/storage/logs
            - ./temp:/var/www/html/storage/app/temp
        ports:
            - "9191:80"
            - "3000:3000"
        networks:
            - linguacafe
        extra_hosts:
            - "host.docker.internal:host-gateway"
    mysql:
        image: mysql:8.0
        container_name: linguacafe-database
        restart: unless-stopped
        tty: true
        ports:
            - "3308:3306"
        volumes:
            - ./mysql:/var/lib/mysql
        environment:
            MYSQL_DATABASE: linguacafe
            MYSQL_USER: linguacafe
            MYSQL_PASSWORD: linguacafe
            MYSQL_ROOT_PASSWORD: linguacafe
            SERVICE_NAME: linguacafe
        healthcheck:
            test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
            interval: 15s
            timeout: 15s
            retries: 8
        command: mysqld --general-log=1 --general-log-file=/var/lib/mysql/general-log.log
        networks:
            - linguacafe
    python:
        container_name: linguacafe-python-service
        command: /app/manage.py runserver 0.0.0.0:8678
        restart: unless-stopped
        tty: true
        image: ghcr.io/simjanos-dev/linguacafe-python-service:main
        volumes:
            - ./dict:/var/www/html/storage/app/dictionaries
            - ./temp:/var/www/html/storage/app/temp
        networks:
            - linguacafe
#        platform: linux/amd64
simjanos-dev commented 6 months ago

It does seem to be correct.

What do you mean by installing it a second time? Is it possible that you did not delete the contents of the ./mysql folder before tried to install it a second time, and it tried to use the old database with the old password, which is used in ./mysql folder?

Also, I think (I don't have a complete understanding of docker) if you don't delete the container, it's not enough to just edit the docker-compose.yml, because it would still use the old one.

If the problem is one of these, or both, then I think stopping your server, deleting everything from inside the ./mysql folder, then running this command:

docker compose up -d --force-recreate

should fix the database access issue.

TheNomad11 commented 6 months ago

Thanks so much! That was it. First the "standard error" with the missing table showed up and the "standard solution" with docker compose down and docker compose up -d solved the issue :) and the login page showed up, thanks so much, looking forward to exploring your app!

simjanos-dev commented 6 months ago

I'm glad it worked, sorry for the messy install process. I'm not good with docker, and we're working on fixing the last problems with the new docker install method.

If you tried it out and have some feedback, please leave a comment/message somewhere. I've already received many, and working on a big update to fix every issue and add new features.

Thank you for trying it out!

TheNomad11 commented 6 months ago

Thanks will do, so far I have been struggling with making dictionaries and deepl work. Imported and activated, but no translations show up, I will try again tomorrow and open issues. For example

screen-2024-01-19-22-26-40

I also wonder where to set the language or is it detected automatically. No api calls to Deepl by the way

simjanos-dev commented 6 months ago

On the bottom left you can select a language, there is no need to select it anywhere else.

If you click on the pen on the popup vocabulary box, you should see the translations search results, which you can add and save for the words. This is one of the confusing and tedious things, which I've received feedback on. The dictionary search will be moved to the main area of the popup box.

DeepL API calls go through back-end, so you won't see it in your browser network stats.

(If you have any more problems I'm happy to explain things and help. The next update should fix most of these issues. Please comment on #73)

TheNomad11 commented 6 months ago

Ah, there, ok, then it works! There was a Japanese flag, and I thought it was just decoration, stupid me! Thanks again!

simjanos-dev commented 6 months ago

@sergiolaverde0 I have tested the new process, and I managed to install a fresh version without any problems.

However after I copied my database folder, and ran the chmod command with root permissions, I've got these error messages in my database container which does not turn green:

2024-01-20 22:15:57 chown: changing ownership of '/var/lib/mysql/#innodb_redo/#ib_redo18_tmp': Operation not permitted
2024-01-20 22:15:57 chown: changing ownership of '/var/lib/mysql/#innodb_redo/#ib_redo20_tmp': Operation not permitted
2024-01-20 22:15:57 chown: changing ownership of '/var/lib/mysql/#innodb_redo/#ib_redo39_tmp': Operation not permitted
2024-01-20 22:15:57 chown: changing ownership of '/var/lib/mysql/#innodb_redo/#ib_redo34_tmp': Operation not permitted

I've also updated the readme file in the dev branch with a new migration process. I thought this would be easier to follow. I've asked people on discord how difficult it is, i'm waiting for a response.

Do you have any idea what the problem is?

If we manage to find out what it is, and manage to do a migration test, v0.5 can be published. If you could also make a final test with the :dev image after we find out what is wrong, it would be great.

I also found this. He is the second person who tried to change the MySQL password. I'll add a note to the readme about it, and think about adding custom password support to MySQL.

simjanos-dev commented 6 months ago

I have fixed it by adding a sudo chown -R $USER:$USER ./database/ command to the process. I've tested it twice, without any problem.

The only thing left is your approval, then it can be released. :) Thank you for the insane amount of help!

sergiolaverde0 commented 6 months ago

TL;DR: I couldn't reproduce, and the error message you got makes me suspect you either did chown at some point or had the line with user: 1000:1000 in the composefile and that messed up with the permissions.

I know it is getting late in your timezone but please attempt to reproduce the error, and if you find it again then send me a copy of the ownership and permissions in your database folder before running chown.

To test, I have ran the version from main with its composefile, only mounts were changed:

version: "3"
name: linguacafe

networks:
    linguacafe:
        driver: bridge

services:
    webserver:
        image: ghcr.io/simjanos-dev/linguacafe-webserver:main
        container_name: linguacafe-webserver
        depends_on:
            mysql:
                condition: service_healthy
        volumes:
            - ./dict:/var/www/html/storage/app/dictionaries
            - ./logs:/var/www/html/storage/logs
            - ./temp:/var/www/html/storage/app/temp
        ports:
            - "9191:80"
            - "3000:3000"
        networks:
            - linguacafe
        extra_hosts:
            - "host.docker.internal:host-gateway"
    mysql:
        image: mysql:8.0
        container_name: linguacafe-database
        restart: unless-stopped
        tty: true
        ports:
            - "3308:3306"
        volumes:
            - ./database:/var/lib/mysql
        environment:
            MYSQL_DATABASE: linguacafe
            MYSQL_USER: linguacafe
            MYSQL_PASSWORD: linguacafe
            MYSQL_ROOT_PASSWORD: linguacafe
            SERVICE_NAME: linguacafe
        healthcheck:
            test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
            interval: 15s
            timeout: 15s
            retries: 8
        command: mysqld --general-log=1 --general-log-file=/var/lib/mysql/general-log.log
        networks:
            - linguacafe
    python:
        container_name: linguacafe-python-service
        command: /app/manage.py runserver 0.0.0.0:8678
        restart: unless-stopped
        tty: true
        image: ghcr.io/simjanos-dev/linguacafe-python-service:main
        volumes:
            - ./dict:/var/www/html/storage/app/dictionaries
            - ./temp:/var/www/html/storage/app/temp
        networks:
            - linguacafe
        # platform: linux/amd64

All folders except the database one were created before starting the containers. Then populated the database by creating an admin user, importing dictionaries, creating a "book" from a news site, reading some of them, and highlighting some words.

After taking the container down, this was the ownership of the database folder:

drwxr-xr-x 1     999 root          672 ene 20 18:02  .
drwxr-xr-x 1 desktop desktop        86 ene 20 18:02  ..
-rw-r----- 1     999     999      1579 ene 20 18:02  6cc8b4c6dc55.err
-rw-r----- 1     999     999        56 ene 20 18:02  auto.cnf
-rw-r----- 1     999     999   3040623 ene 20 18:02  binlog.000001
-rw-r----- 1     999     999 211103148 ene 20 18:12  binlog.000002
-rw-r----- 1     999     999        32 ene 20 18:02  binlog.index
-rw------- 1     999     999      1680 ene 20 18:02  ca-key.pem
-rw-r--r-- 1     999     999      1112 ene 20 18:02  ca.pem
-rw-r--r-- 1     999     999      1112 ene 20 18:02  client-cert.pem
-rw------- 1     999     999      1680 ene 20 18:02  client-key.pem
-rw-r----- 1     999     999 447779296 ene 20 18:12  general-log.log
-rw-r----- 1     999     999    196608 ene 20 18:12 '#ib_16384_0.dblwr'
-rw-r----- 1     999     999   8585216 ene 20 18:07 '#ib_16384_1.dblwr'
-rw-r----- 1     999     999      5676 ene 20 18:02  ib_buffer_pool
-rw-r----- 1     999     999  12582912 ene 20 18:12  ibdata1
-rw-r----- 1     999     999  12582912 ene 20 18:02  ibtmp1
drwxr-x--- 1     999     999       688 ene 20 18:07 '#innodb_redo'
drwxr-x--- 1     999     999       202 ene 20 18:02 '#innodb_temp'
drwxr-x--- 1     999     999       798 ene 20 18:03  linguacafe
drwxr-x--- 1     999     999       178 ene 20 18:02  mysql
-rw-r----- 1     999     999  32505856 ene 20 18:12  mysql.ibd
lrwxrwxrwx 1     999     999        27 ene 20 18:02  mysql.sock -> /var/run/mysqld/mysqld.sock
drwxr-x--- 1     999     999      5032 ene 20 18:02  performance_schema
-rw------- 1     999     999      1680 ene 20 18:02  private_key.pem
-rw-r--r-- 1     999     999       452 ene 20 18:02  public_key.pem
-rw-r--r-- 1     999     999      1112 ene 20 18:02  server-cert.pem
-rw------- 1     999     999      1680 ene 20 18:02  server-key.pem
drwxr-x--- 1     999     999        28 ene 20 18:02  sys
-rw-r----- 1     999     999  16777216 ene 20 18:12  undo_001
-rw-r----- 1     999     999  16777216 ene 20 18:12  undo_002

After running sudo chmod 777 -R ./ the ownership hadn't changed, only permissions did.

Then I cloned the deploy branch and ran chmod 777 -R ./ , no need for sudo. Then finally copied the contents of the main folders to the deploy folders. Ran docker compose up -d and everything went fine for me, and no data that was stored in the database was lost.

simjanos-dev commented 6 months ago

It is interesting. My user name is laptop. This is a FinalV04Mysql folder where I copied my v0.4 database as a backup, to test the migration process multiple times.

So I copied: old database directory -> FinalV04Mysql -> new database directory.

laptop@laptop-mrcwx0:~/Projects/FinalV04Mysql$ ls -lah
total 1,1G
drwxr-xr-x  8 root   root   4,0K jan   20 15:20  .
drwxrwxrwx 10 laptop laptop 4,0K jan   20 17:49  ..
-rw-r-----  1 root   root     56 jan   20 15:19  auto.cnf
-rw-r-----  1 root   root   2,9M jan   20 15:19  binlog.000001
-rw-r-----  1 root   root   320M jan   20 15:19  binlog.000002
-rw-r-----  1 root   root    19M jan   20 15:20  binlog.000003
-rw-r-----  1 root   root   4,5K jan   20 15:19  binlog.000004
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000005
-rw-r-----  1 root   root   216K jan   20 15:19  binlog.000006
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000007
-rw-r-----  1 root   root   168M jan   20 15:20  binlog.000008
-rw-r-----  1 root   root    28M jan   20 15:20  binlog.000009
-rw-r-----  1 root   root   161M jan   20 15:20  binlog.000010
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000011
-rw-r-----  1 root   root    49M jan   20 15:20  binlog.000012
-rw-r-----  1 root   root   7,9K jan   20 15:20  binlog.000013
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000014
-rw-r-----  1 root   root    956 jan   20 15:19  binlog.000015
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000016
-rw-r-----  1 root   root   580K jan   20 15:20  binlog.000017
-rw-r-----  1 root   root   265M jan   20 15:20  binlog.000018
-rw-r-----  1 root   root   5,7M jan   20 15:20  binlog.000019
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000020
-rw-r-----  1 root   root    23M jan   20 15:20  binlog.000021
-rw-r-----  1 root   root   7,1K jan   20 15:20  binlog.000022
-rw-r-----  1 root   root    180 jan   20 15:20  binlog.000023
-rw-r-----  1 root   root   2,7K jan   20 15:20  binlog.000024
-rw-r-----  1 root   root    384 jan   20 15:20  binlog.index
-rw-------  1 root   root   1,7K jan   20 15:19  ca-key.pem
-rw-r--r--  1 root   root   1,1K jan   20 15:19  ca.pem
-rw-r--r--  1 root   root   1,1K jan   20 15:19  client-cert.pem
-rw-------  1 root   root   1,7K jan   20 15:19  client-key.pem
-rw-r-----  1 root   root   1,6K jan   20 15:19  d039b00624cb.err
-rw-r-----  1 root   root   192K jan   20 15:19 '#ib_16384_0.dblwr'
-rw-r-----  1 root   root   8,2M jan   20 15:19 '#ib_16384_1.dblwr'
-rw-r-----  1 root   root   5,2K jan   20 15:20  ib_buffer_pool
-rw-r-----  1 root   root    12M jan   20 15:19  ibdata1
drwxr-x---  2 root   root   4,0K jan   20 15:19 '#innodb_redo'
drwxr-x---  2 root   root   4,0K jan   20 15:19 '#innodb_temp'
drwxr-x---  2 root   root   4,0K jan   20 15:20  linguacafe
drwxr-x---  2 root   root   4,0K jan   20 15:19  mysql
-rw-r-----  1 root   root    31M jan   20 15:19  mysql.ibd
lrwxrwxrwx  1 root   root     27 jan   20 15:19  mysql.sock -> /var/run/mysqld/mysqld.sock
drwxr-x---  2 root   root   4,0K jan   20 15:19  performance_schema
-rw-------  1 root   root   1,7K jan   20 15:19  private_key.pem
-rw-r--r--  1 root   root    452 jan   20 15:19  public_key.pem
-rw-r--r--  1 root   root   1,1K jan   20 15:19  server-cert.pem
-rw-------  1 root   root   1,7K jan   20 15:19  server-key.pem
drwxr-x---  2 root   root   4,0K jan   20 15:19  sys
-rw-r-----  1 root   root    16M jan   20 15:19  undo_001
-rw-r-----  1 root   root    16M jan   20 15:19  undo_002

I'm not sure what happened. Keeping the chown command wouldn't hurt, but also don't want to add unnecessary commands.

I'm still gonna be up for a while, but I think i'll only run a test in the morning. If it works for you, it is probably something I messed up.

sergiolaverde0 commented 6 months ago

If you manage to run a new test without encountering the issue I recommend to leave the extra command out of the regular migration instructions, but keep it handy if someone asks you or maybe even add it to a troubleshooting section. If you can reproduce it then keep it as part of the regular instructions to go on the safe side.

simjanos-dev commented 6 months ago

Okay. After I done that and either kept it or changed it, can I release it when I wake up? I think everything's ready.

sergiolaverde0 commented 6 months ago

Yes, go ahead.

simjanos-dev commented 6 months ago

I've had no issues on my second attempt to migrate. I'll added an extra line to the readme to be safe. I'll merge in the new update now.

simjanos-dev commented 6 months ago

I have noticed that the python image is downloaded more than the php. It was ahead of php in download count by 2, now only by one.

Any idea how or why? Is it because of a cancelled image pull command?

sergiolaverde0 commented 6 months ago

Most likely is because of that, yes. It could even be that you checked at the specific instant when someone was just starting a pull and it began with the python image; that would explain why the gap closed a little afterwards, as the second image began to be pulled.

Edit: I gotta say, these post-release hours seem to be quite uneventful for how many changes were made.