CDN: Phase I: Shared product data module.

This will be done in 3 phases.

Phase I: Shared data (commerce) will be imported via a @luxfi/data module.

Phase II: All assets referenced in that data will be served from cdn.lux.network (prod) or something like localhost:3001 in dev

Phase III: More advanced. See thread below: (June 4, 2024) Z: i had planned to create a @luxfi/data or @luxfi/assets folder which would have all the shared assets in it, and then just serve that from the cdn site, but rn all the files are just in public in cdn/public and served from cdn.lux.network you can just add more files there / update that. A: One question: I’m assuming that site specific stuff (not commerce images) are still served from the individual sites or ? Z: yeah site specific stuff like credit-specific background images and shit should just be in the credit/public folder anything that's shared should go into cdn ... if you want to be a bit fancier you can add this to cdn:

app/[...path]/route.ts

import { NextRequest, NextResponse } from 'next/server';
import path from 'path';
import fs from 'fs';
import { createRequire } from 'module';

const require = createRequire(import.meta.url);

export async function GET(req: NextRequest, { params }: { params: { path: string[] } }) {
  const { path: filePathArray } = params;

  try {
    // Get the directory of the @luxfi/data package
    const luxfiDataPath = path.dirname(require.resolve('@luxfi/data/package.json'));

    // Construct the file path within the @luxfi/data package
    const filePath = path.join(luxfiDataPath, ...filePathArray);

    if (!fs.existsSync(filePath)) {
      return NextResponse.json({ error: 'File not found' }, { status: 404 });
    }

    const fileContent = fs.readFileSync(filePath);
    const fileExtension = path.extname(filePath).substring(1);
    const contentType = getContentType(fileExtension);

    return new NextResponse(fileContent, {
      headers: {
        'Content-Type': contentType,
      },
    });
  } catch (error) {
    return NextResponse.json({ error: 'Internal server error' }, { status: 500 });
  }
}

function getContentType(extension: string) {
  const mimeTypes: { [key: string]: string } = {
    'html': 'text/html',
    'js': 'application/javascript',
    'json': 'application/json',
    'css': 'text/css',
    'png': 'image/png',
    'jpg': 'image/jpeg',
    'gif': 'image/gif',
    'svg': 'image/svg+xml',
    'pdf': 'application/pdf',
    'txt': 'text/plain',
    'xml': 'application/xml',
  };

  return mimeTypes[extension] || 'application/octet-stream';
}

and then that would basically just make the cdn route return files from @luxfi/data package for anythning i think you have to change next.config.js to this:

// next.config.js
module.exports = {
  async rewrites() {
    return [
      {
        source: '/:path*',
        destination: '/api/:path*', // Proxy to dynamic route
      },
    ];
  },
};

something like that for the new app router should work

if you want to be super fancy you can use unpkg proxy

something like:

import { NextRequest, NextResponse } from 'next/server';
import fetch from 'node-fetch';

export async function GET(req: NextRequest, { params }: { params: { version: string, path: string[] } }) {
  const { version, path: filePathArray } = params;

  // Construct the unpkg URL
  const unpkgUrl = `https://unpkg.com/@luxfi/data@${version}/${filePathArray.join('/')}`;

  try {
    // Fetch the file from unpkg
    const response = await fetch(unpkgUrl);

    if (!response.ok) {
      return NextResponse.json({ error: 'File not found' }, { status: 404 });
    }

    const fileContent = await response.buffer();
    const contentType = response.headers.get('content-type') || 'application/octet-stream';

    return new NextResponse(fileContent, {
      headers: {
        'Content-Type': contentType,
      },
    });
  } catch (error) {
    return NextResponse.json({ error: 'Internal server error' }, { status: 500 });
  }
}

then you could reference specific luxfi/data versions to cdn and get the right stuff back that's kind of the ultimate because then you never lose any old asset version they all exist on npm and apps can even depend on fixed asset versions w/o worrying about stuff going away later

// corresponding next.config.js for unpkg magic
module.exports = {
  async rewrites() {
    return [
      {
        source: '/:version/:path*',
        destination: '/api/:version/:path*', // Proxy to dynamic route
      },
    ];
  },
};

would be cool to basically serve latest deployed @luxfi/data from cdn.lux.network/commerce/etc but then also support cdn.lux.network/v1.0.0/commerce to trigger proxy to unpkg oh if you do add unpkg support make sure to do this tho:

    headers.set('Cache-Control', 'public, max-age=31536000, immutable'); // Cache forever

since they will never return a diff file for a specific version it should cache it forever, that will make sure the backend dynamic route only runs 1 time to return the file and should be cached on edge forever after that anyways thanks for tackling this! i'm going back to bed lol 2:54 PM Z i think this can also go away: https://github.com/luxfi/web/tree/main/assets

i would also move all the product config stuff like the stuff you were generating to packages/data/commerce/products.json

or whatnot so they can all be imported in a unified way from cdn like:

await import('https://cdn.lux.network/commerce/data.json')

in all commerce layer so all commerce data is unified and imported async from cdn dynamically at runtime for each site. alternately we can just import @luxfi/data or whatnot and keep it all there and then it gets built in to each bundle. right now if there is less than 50kb of commerce data you can probably just bundle it in i guess, long term it's better to load it on client and then you can also update the commerce data independently of re-deploying the site updating all product stuff then becomes as easy as deploying latest to cdn... altho the way hanzo does this we actually invalidate the cache on cloudflare, not sure how to do this with vercel...normally when we update product info in backend or via API it actually invalidates the cloudflare cache so you can update the product info but it stays cached at edge constantly w/o expiring oh i think vercel actually DOES purge automatically on new deploy so it would just work automagically

luxfi / web

CDN: Phase I: Shared product data module. #82