finaldie / auto-news

A personal news aggregator to pull information from multi-sources + LLM (ChatGPT/Gemini/Ollama via LangChain) to help us reading efficiently with less noises, the sources including: Tweets, RSS, YouTube, Web Articles, Reddit, and personal Journal notes.
MIT License
502 stars 55 forks source link

Impossible to install autonews on Macbook M1 ? #90

Open aurlro opened 2 months ago

aurlro commented 2 months ago

I choose Docker Installation on my mac M1 but each time i found many errors ... processeur amd64 replace on code by arm64, i tried to fix it with chatgpt but each time I can't open mysql database and i'm blocked.

finaldie commented 2 months ago

Hi @aurlro

processeur amd64 replace on code by arm64

Does that mean you change the processor after installation? Maybe you could share some error details.

aureseth commented 2 months ago

Excuse me, I expressed myself badly. With the help of chatgpt I was able to try to install and set up the .yalm file to add the arm64 Linux platforms (which seems to be suitable for Apple's M1, M2 etc processor). Unfortunately, I still haven't managed to do that so far. The Rosettat system does not work miracles this time.

finaldie commented 2 months ago

From docker installation wiki, we actually don't need to touch the .yaml file. Instead, the build/.env is the target file to be prepared. All the platform-related stuff will be covered by Docker itself.

Trying to understand in which step you were blocked and I can help there.

May I know which yaml file you modified (with changes), and what errors you were facing? And please don't share any private keys, e.g. openai API keys, notion tokens, etc.

aurlro commented 2 months ago

To start from scratch, I've just emptied docker and started again. The blocking step is the following command:

‘sh-3.2# make start

Next, here's what happens :

creating docker network: bot_network... docker network inspect bot_network >/dev/null 2>&1 || docker network create bot_network cd docker && make start topdir=/Users/aurelien/autonews docker-compose --env-file ../build/.env up -d WARN[0000] The ‘AIRFLOW_UID’ variable is not set. Defaulting to a blank string. WARN[0000] /Users/aurelien/autonews/docker/docker-compose.yaml: the attribute version is obsolete, it will be ignored, please remove it to avoid potential confusion

! airflow-init The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8[+] Running 21/22 platform was requested 0.0s ! milvus-ui The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s ! airflow-webserver The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s ! airflow-triggerer The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s ! airflow-scheduler The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s ! airflow-worker The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s ✔ Container docker-airflow-init-user-1 Created0.3s ! airflow-init-user The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s dependency failed to start: container mysql-db is unhealthy make[1]: [start-services] Error 1 make: [start] Error 2 ✘ Container mysql-db Error64.5s

Services lancées$@2x

finaldie commented 2 months ago

Let me get a M1 macbook and try to reproduce the issue.

Sacha338 commented 2 months ago

I just installed it and here are the commands I ran (I didn't get any errors):

git clone https://github.com/finaldie/auto-news.git

cd auto-news

docker compose -f docker/docker-compose.yaml up -d (A bit long)

docker compose -f docker/docker-compose.yaml ps

And then I went to http://localhost:8080

finaldie commented 2 months ago

@Sacha338 , Thanks for providing more details, in the meantime, if possible, do you mind checking the MySQL docker instance log? I'd wondering whether there are some error logs in your previous deployment.

finaldie commented 2 months ago

@Sacha338

I reproduced this issue and found that MySQL recently released major version 9, which is incompatible with MySQL-8 parameters. I have fixed MySQL to 8.4.2.

Please use the below steps to pick the fixes (assume the build/.env is ready):

1. make stop
2. rm -rf workflow
3. git pull
4. make deps && make deploy
5. make start

Let me know if any issues, thanks.

aurlro commented 2 months ago

After all these corrections, I end up with a similar problem Docker

https://github.com/user-attachments/assets/dcf503e5-7416-4cd0-9357-bed41f106d5c

Sacha338 commented 2 months ago

@Sacha338 , Thanks for providing more details, in the meantime, if possible, do you mind checking the MySQL docker instance log? I'd wondering whether there are some error logs in your previous deployment.

How ? Where can I found the logs

finaldie commented 2 months ago

@Sacha338

Thank you for uploading the screenshots; it narrows down the problem.

So far, I have tested different machines; some work, some don't, especially from MacBook (upgraded to the latest Docker desktop). I found that the docker compose cannot pick up the --env-file parameter, which causes the env file to have no effect. To recap, it impacts the docker-compose installation (for Mac, some docker versions); the portainer/k8s setup has no issues.

Because this is related to the docker compose command, I don't exactly know how many versions (from when) have the issue. I'm thinking about copying the build/.env into the runtime environment. It may not be the perfect solution, but it could fix this issue regardless of the platform and the version of docker-compose we use.

I'll create a new PR to address it then.

finaldie commented 2 months ago

@Sacha338

Please try the latest fix by following the following steps. These will copy the build/.env into the runtime docker container without relying on the docker compose parameter.

1. make stop
2. git pull
3. make deps && make deploy
4. make start

Btw, I noticed that M1 chip is not that strong, we may have to wait a few minutes to get the airflow ready.

aurlro commented 2 months ago

For the first time, notion has created the auto-news structure! So we're moving forward! On Docker, I have 12 services that are ALL activated. Unfortunately, I think I still have errors because the new pages created automatically are usually empty. Can you create a standard scenario to follow to check that everything's working? In subsequent updates, can you include error logs so that you can then correct them? New page 2@2x Docker@2x

finaldie commented 2 months ago

Journal will be empty notes if no user's daily inputs, this is normal and means it's working.

The page structure is like this:

Readings
├── Inbox
│   ├── Inbox - Article
│   └── Inbox - YouTube
│   └── Inbox - Journal
├── Index
│   ├── Index - Inbox
│   ├── Index - ToRead
│   ├── RSS_List
│   └── Tweet_List
│   └── Reddit_List
└── ToRead
    └── ToRead

After installation, the Index/RSS_List and Index/Reddit_List are empty, add your favorite RSS and subreddits there, then the posts will be feed in later.

Examples: