11ty / eleventy-fetch

Utility to cache any remote asset: Image, Video, Web Font, CSS, JSON, etc
https://www.11ty.dev/docs/plugins/fetch/
144 stars 19 forks source link

Is it possible to Cache an array of URLs? #9

Closed tannerdolby closed 3 years ago

tannerdolby commented 3 years ago

I'm using eleventy-cache-assets to fetch repository JSON data from the Github API using the URL https://api.github.com/repos/tannerdolby/eleventy-photo-gallery.

Utilizing a global data file _data/gitPhotoGallery.js, everything works great to Cache the data from api but how can I Cache multiple repositories data? I'm currently creating a new data file for each new repository I want to get api data for. (Adding stars and some other fields to the projects section of my site, which is about 4 repositories needing api requests to cache the json result)

With my current setup, I'm creating a separate global data file to cache the API result from github for a new repo. If _data/gitPhotoGallery.js links to the request for api.github...tannerdolby/eleventy-photo-gallery then I would create a new data file to fetch JSON data about my website's repo api.github...tannerdolby/tannerdolby.com in _data/gitTannerDolby.js.

const Cache = require("@11ty/eleventy-cache-assets");

module.exports = () => {
    // without try/catch block for length purposes
    let json = Cache("https://api.github.com/repos/tannerdolby/eleventy-photo-gallery", {
        duration: "1d",
        type: "json"
    };
    return {
        repoName: json.name,
        githubHandle: json.owner.login,
        homepage: json.homepage,
        stargazers: json.stargazers_count,
        watchers: json.watchers_count
    }
};

I understand the usage is const Cache: (source: any, opts: any) => Promise<any> and since it returns a single Promise it only makes sense that a single file ie github.js is only capable of caching one repository worth of JSON data from the github API.

Can I cache multiple assets and return a single Promise or do I need to create separate global data files for each new repository I want to make an API call for? From my understanding of Promise, I think I might be asking a silly question but wanted to understand better. Any info would be greatly appreciated! @zachleat

tannerdolby commented 3 years ago

I figured out a method for handling what I needed. To avoid creating multiple global JS data files for caching each github repositories API data, I put together an async function fetchData(url) that accepts one parameter, the URL string for sending a request to the Github API and returns a Promise which will contain the API data as <Promise{ title: "bla, desc: "bla", .. }>. Which can then be accessed in a template like normal.

Below is the code within _data/github.js

const Cache = require("@11ty/eleventy-cache-assets");

// Cache the github API request url passed in 
async function fetchData(url) {
    console.log("Attempting to Cache API request");
    try {
        let json = await Cache(url, {
            duration: "1d",
            type: "json",
        });
        return {
            title: json.name,
            desc: json.description,
            stargazers: json.stargazers_count,
            subscribers: json.subscribers_count,
            forks: json.forks_count,
            issues: json.open_issues,
            stargazers_url: json.stargazers_url,
            homepage: json.homepage,
            repo_url: json.html_url
        }
    }
    catch (e) {
        console.log("Error caching Github API data");
        return {
            title: "A Github Project by Tanner",
            desc: "This project was created by @tannerdolby",
            stargazers: 0,
            subscribers: 0,
            forks: 0,
            issues: 0,
            stargazers_url: "https://github.com/tannerdolby",
            homepage: "https://github.com/tannerdolby",
            repo_url: "https://github.com/tannerdolby"
        }
    }
}

module.exports = async function() {
    try {
        const eleventyGallery = await fetchData("https://api.github.com/repos/tannerdolby/eleventy-photo-gallery");
        const personalWebsite = await fetchData("https://api.github.com/repos/tannerdolby/tannerdolby.com");

       // return the promise for each project <Promise{ title: ... }>
        return { eleventyGallery, personalWebsite };
    } catch (e) {
        console.log("Error returning multiple projects cached API data");
    } 
};

Using the data in a template works as expected! Allowing multiple github repositories data to be accessed from one global JS data file thanks to Caching the API request and returning the promise.

{{ github.eleventyGallery.title }} // outputs 'eleventy-photo-gallery'
{{ github.eleventyGallery.stargazers }} // outputs 29

{{ github.personalWebsite.title }} // outputs 'tannerdolby.com'
{{ github.personalWebsite.stargazers }} // outputs 1

I'm happy with the current implementation. By the way, great work on this plugin.

tannerdolby commented 3 years ago

Note: Even though I've found a way to fetch multiple github repositories data in a single global data file to be used in a template, the question of "is it possible to cache an array of URLs" still stands.

I understand its not currently allowed to pass an array of api request URLs to eleventy-cache-assets as it returns a single <Promise>:

module.exports = async () => {
    let json = await Cache(["url-one", "url-two", "url-three"], {
        duration: "1d",
        type: "json"
    }
    // I think this only returns a single promise since `Cache()` accepts a 
    // single URL for the first parameter and an object as the second.
    return {
        title: json.title
    }
}

Would this feature be worth looking into? Allowing the Cache to accept an array of URLs and return multiple promises to be used in templates after returned from module.exports. It might not be possible without creating separate Cached API requests for each URL and then using the returned promise like I did with fetchData(url). Sorry for the rant!

tannerdolby commented 3 years ago

Closing for now, as I've accomplished caching multiple github API requests in a single global js data file. May be reopened if need be.