Welcome to the El Turrero Post project! This website is designed to showcase the Twitter threads of Javier G. Recuenco who specializes in complexity science. The goal of the website is to present the he's tweets in a visually pleasing and easy-to-navigate format.
The website is built using:
The front-end is located at the root of the project folder and the scraping logic is located under the scripts
folder on its own package.
To get started with the project, you will need to clone the repository and install the dependencies. Here are the steps:
git clone git@github.com:karliky/turrero.git
npm install
npm run dev
http://localhost:3000
The process of adding a new thread is half manual half automated:
You could use the script: $ ./scripts/add_thread.sh $id $first_tweet_line
where id
is the first tweet id (thread id) and first_tweet_line
is the first tweet text.
Alternatively you could use the following steps:
$ node ./scripts/add-new-tweet.js $id $first_tweet_line
to add the first twwet id (thread id) and the first tweet text to ontop of the turras.csv file$ node ./scripts/recorder.js
. This will scrap it and save it into tweets.json$ node ./scripts/tweets_enrichment.js
$ node ./scripts/image-card-generator.js
./scripts/metadata
content into public/metadata
, you can use the following command mv -v ./metadata/* ./public/metadata/
$ node ./scripts/make-algolia-db.js
then update the index in the Algolia service, clear the index and fetch the db/tweets-db.json
file$ node ./scripts/generate-books.js
this will update the db/books-not-enriched.json
$ node ./scripts/book-enrichment.js
this will output the list of books you should use in the last prompt to get the category of books and update the db/books.json
$ ./scripts/generate_prompts.sh
to generate the prompts to be used with ChatGPT (or the LLM of your choice) and obtain the summary, categories and questions to include in db/tweets_summary.json
, db/tweets_map.json
and db/tweets_exam.json
respectivelycomponents/header.js
to the latest update datenpm run dev
on the root folderThe files db/tweets.json, db/tweets-db.json, db/tweets_enriched.json
are automatically generated and should not be manually edited.
The files db/tweets_map.json, db/tweets_summary.json
, db/tweets_exam.json
should be manually edited.
We welcome contributions to this project. If you find any bugs or have any suggestions for new features, please open an issue or a pull request on the GitHub repository.