search

bogeeee / restfuncs

MIT License
35 stars 1 forks source link

readme

Restfuncs - HTTP API done proper

What is Restfuncs: Coming from tRPC Restfuncs is also an RPC (Remote Procedure Call) library. It also has a client and sever part and also gives the user end2end type safety. The key differences are: - Restfuncs takes [tRPCs claim: "It's just functions"](https://trpc.io/docs/concepts#its-just-functions) for real also for the server side 😎, resulting in a much simpler usage and way less boilerplate. - Restfuncs uses (native) typescript for validation, instead of ZOD or other type generators. - It has websocket support as first class citizen, enabled by default. Therefore push events can be achieved just via plain callback functions 👍 which you can pass anywhere in the arguments. - Cookie sessions / JWT, CSRF protections and CORS are tightly integrated into Restfuncs instead of beeing 3rd party plugins or having to be coded manually. _This was especially necessary for supporting / syncing with websockets._ - Concepts are simplified, aiming for less total boilerplate and a more shallow learning curve. - Also see the additional features in the [list](#intro--features). Here's a mapping of the tRPC's conceptual items to Restfuncs: - `Procedure`/`Query`/`Mutation` -> No distinctions between them in restfuncs. It's all just @remote methods. Following GET/POST semantics is done by a client (if needed) and Restfuncs [serves just both styles](#rest-interface) instead of needing configuration. - `Router` -> `ServerSession` - `Context` -> `ServerSession` - _you can also store fields there._ - Middlewares -> You just overwrite the ServerSession#doCall method for this (use ctrl+space / intellisense in your IDE). Needs no further explanation or docs, if you know some basic OOP concepts. - `Subscriptions` -> You can simply use callback functions anywhere in your @remote methods. _When called, the events automatically get send to the client via websocket push events. No need to set up that channel or synchronize that context manually 👍_. - Inferring Types -> Not needed. Can just be achieved by Typescript.
What is Restfuncs: Coming from Nest or other "big frameworks" Restfuncs is not a framework for organizing your code (and does not want to be such). It is just a small layer above express, to improve communication needs in a one-tool-for-one-purpose manner. Similiar to those frameworks, it makes coding API endpoints easier and offers you a rich set of features around those (but, as said, just what tightly belongs to http communication. Nothing else). Also, Restfuncs it is made for RPC (Remote Procedure Calls): That means, on the client, you don't code a fetch request by hand, but just call a remote method, as if it was a normal javascript method (but across the wire). It's an old concept in the IT that was a bit forgotten but regains traction again in the JS world, cause it makes especially sense in a scenario with typescript, both, on the client and the server side. Therefore, all your calls can be checked for type safety at compile time (end2end type safety). A similar popular library that offers such a concept is [tRPC](https://trpc.io/). Think of Restfuncs as a more modern alternative of that. A Nest' "Controller" (=Service) corresponds to a "ServerSession" class in Restfuncs. That's the only organization unit, it has. Wiring them to the express routes is done manually. Now you put a few methods (=endpoints) and fields (=session cookie fields) into such a ServerSession class and that's already all the concept ;) There's really nothing more conceptually. Just a lot of configuration options around that.
What is Restfuncs: Coming from Express or new to Javascript Coding Express handlers (let's call them "endpoints" now) is fun, but soon you will notice, that this is pretty low level and you do all that repetitive tasks over and over again, like: - Converting from request/body parameters into local variables - Checking these variables against evil input - Doing error handling and reporting The same thing on the client where, each time, you code a fetch request by hand and do conversion, status checking/error handling,... Now, instead of a request handler, in Restfuncs you code your endpoint as a plain javascript function (with typescript types). And it also offers a client, where you can just virtually call that same function from the client (but it goes over the wire via http or websockets). This It's an old concept in the IT that's called RPC (Remote Procedure Call) and it was a bit forgotten but regains traction again in the JS world, cause it makes especially sense in a scenario with typescript, both, on the client and the server side. Therefore, all your calls can be checked for type safety at compile time (end2end type safety). A similar popular library that offers such a concept is [tRPC](https://trpc.io/). Think of Restfuncs as a more modern alternative of that. But besides RPC, Restfuncs deals with much more aspects around http-communication, that play together and make just sense to be tightly integrated here into this communication library. But see the features for yourself.

Intro + features

With restfuncs, you write your http API endpoints just as plain typescript functions. Or better say: methods.
Per-endpoint boilerplate is basically:

@remote()
greet(name: string) {
    return  `Hello ${name}` 
}

See, it uses natural parameters and natual return and throw flow, instead of dealing with req and res and Restfuncs will take care about a lot more of your daily, low-level communication aspects.
That is (features):

Smaller features:

<Boilerplate cheat sheet - all you need to know>

Security note: When using client certificates, you must also read the CSRF protection chapter.

MyServerSession.ts

import {ServerSession, ServerSessionOptions, UploadFile, remote, ClientCallback} from "restfuncs-server";

export class MyServerSession extends ServerSession {

  static options: ServerSessionOptions = {/* ... */}

  myLogonUserId?: string // This value gets stored in the session-cookie under the key "myLogonUserId".

  /**
   * This JSDoc also gets outputted in the public API browser and OpenAPI spec. Write only nice things here ;)
   * @param myComplexParam Your parameters can be of any typescript type. They are automatically validated at runtime.
   * @param myCallback   You can pass server->client callback functions anywhere/deeply                          . Here we send the progress of the file upload. Callback's args and results are validated 👍. But this works only for "inline"- callbacks, see readme.md.
   * @param myUploadFile You can pass UploadFile objects                anywhere/deeply/also as ...rest arguments. As soon as you read from the the stream, the restfuncs client will send that file in an extra http request in the background/automatically.
   */
  @remote({/* RemoteMethodOptions */})
  myRemoteMethod(myComplexParam: { id?: number, name: string }, myCallback?: (percentDone: number) => void, myUploadFile?: UploadFile) {
    // ADVANCED:
    // this.call.... // Access or modify the current call's context specific properties. I.e. this.call.res!.header("myHeader","...")
    // (myCallback as ClientCallback).options.... // Access some options under the hood

    return `Hello ${myComplexParam.name}, your userId is ${this.myLogonUserId}` // The output automatically gets validated against the declared or implicit return type of `myRemoteMethod`. Extra properties get trimmed off.
  }

  // <-- More @remote methods

  // <-- methods, which serve html / images / binary. See https://github.com/bogeeee/restfuncs#html--images--binary-as-a-result

  // <-- Intercept **each** call, by overriding the `doCall` method. I.e. check for auth (see example project), handle errors. Use your IDE's intellisense (ctrl+space) to override it.
}

server.ts

import {restfuncsExpress} from "restfuncs-server";
import {MyServerSession} from "./MyServerSession.js";

const app = restfuncsExpress({/* ServerOptions */}) // Drop in replacement for express (enhances the original). Installs a jwt session cookie middleware and the websockets listener. Recommended.
app.use("/myAPI", MyServerSession.createExpressHandler())

// Optional: app.use(helmet(), express.static('dist/web')) // Serve pre-built web pages / i.e. by a bundler like vite, parcel or turbopack. See examples. It's recommended to use the helmet() middleware for additional protection.
// Optional: app.use(...) //<-- Serve *other / 3rd party* express routes here. SECURITY: These are not covered by restfuncs CSRF protection. Don't do write/state-changing operations in here ! Instead do them by MyServerSession.

app.listen(3000); // Listen on Port 3000

client.ts

// Use a bundler like vite, parcel or turbopack to deliver these modules to the browser (as usual, also see the example projects): 
import {UploadFile} from "restfuncs-common";
import {RestfuncsClient} from "restfuncs-client";
import {MyServerSession} from "../path/to/server/code/or/its/packagename/MyServerSession.js" // Gives us the full end2end type support

const myRemoteSession = new RestfuncsClient<MyServerSession>("/myAPI", {/* RestfuncsClientOptions */}).proxy; // Tip: For intercepting calls (+ more tweaks), sublcass it and override `doCall`. See the auth example.  

console.log( await myRemoteSession.myRemoteMethod({name: "Hans"}) ); // finally, call your remote method over the wire :)

// And an example call with a callback + a file upload:
const myDomFile = document.querySelector("#myFileInput").files[0]; // Retrieve your File object(s) from an <input type="file" /> (here), or from a DragEvent.dataTransfer.files
await myRemoteSession.myRemoteMethod(...,  (progress) => console.log(`The callback says: ${progress}% uploaded`), myDomFile as UploadFile) // Note: You must cast it here to the server's `UploadFile` type, to resemble Restfuncs's automatic client->server translation.

Setting up the build (here, it gets a bit nasty 😈)

tsconfig.json

"compilerOptions": {
    "moduleResolution": "node",
    "experimentalDecorators": true,
    "strictNullChecks": true,
    "sourceMap": true, //optional, recommended
    "plugins": [
        { "transform": "restfuncs-transformer",  "transformProgram": true},
        { "transform": "typia/lib/transform" },
        { "transform": "typescript-rtti/dist/transformer" } ],
},
"exclude": ["dist", "client", "web"], // Make sure, to not accidentially transform your client files.

package.json

"scripts": {
    "dev": "cross-env NODE_ENV=development <use your favourite tsx / bun / jest / vitest / ...>  #NODE_ENV=development disables all security validations in restfuncs and therefore the need for all the transfomed stuff."
    "clean": "tspc --build --clean",
    "build": "tspc --build --force",
    "start": "cross-env NODE_ENV=production node --enable-source-maps server.js"
},
"dependencies": {
  "restfuncs-server": "^3.0.0",
  "restfuncs-client": "^2.0.0"
},
"devDependencies": {
  "ts-patch": "^3.0.2",
  "restfuncs-transformer": "^1.0.0",
  "cross-env": "^7.0.3"
},

Here we compile with tspc (instead of tsc) from the ts-patch package, which allows for our transformer plugins (No worries: Despite the name "ts-patch", it runs in "live mode" so nothing will be patched here).
See, how the transformer chain works.

</Boilerplate cheat sheet>

Congrats, you've got the concept!
Now use your IDE's intellisense and have a quick browse through the /* XxxOptions */ and also the Callback and UploadFile description/members. That JSDoc is considered the official documentation and it won't be repeated here. In some cases where more configuration needs to be decided for, contextual error messages will guide you. So don't be scared of them and read them and see them as part of the concept.

TODO: Create a more guided documentation (help wanted)





Example projects

They use vite, which is a very minimalistic/ (zero conf) web packer with full support for React/JSX, Typescript, hot module reloading. Hope you'll like this as a starter stack for your webapp.

Advanced

Html / images / binary as a result

To serve a non API result, the remote method must explicitly set the content type. Return the result via string, Buffer or Readable. Example:

    @remote({isSafe: true /* Lessen restrictions and allow this method to be called by GET ... */}) 
    getAvatarImage(name: string) {
        // ... therefore (SECURITY) code in `isSafe` methods must perform read operations only !
        this.call.res!.contentType("image/x-png")
        return fs.createReadStream("/someImage.png") // Returns a Readable which is streamed to client. You can also return Buffer, String, File(TODO)
    }

Security note: When serving html with rich content or with scripts, you might want to add the helmet middleware in front of your ServerSession for additional protection via app.use("/myAPI", helmet(), MyServerSession.createExpressHandler())

REST interface

Security note: For handcrafted calls from inside a browser, you (the clients) need to care about protecting your session from CSRF.

Tl;dr: Just form the http call from your imagination, and its likely a way that works, or Restfuncs will tell you exactly, what's wrong with the params.

Now to the content:
Like the name Restfuncs suggests, there's also a REST/http interface for the case that you don't use the neat RestfuncsClient, or you want to call these from non-js languages, etc.
Restfuncs follows a zero conf / gracefully accepting / non-strict approach (a client is still free to implement strictness to the REST paradigm):
The following example remote method...

    async getBook(name: string, authorFilter?: string) {

    }

...can be called in almost every imaginable way through http like:

Method Url Body Description
GET /getBook/1984/George%20Orwell List arguments in the path
GET /getBook?1984,George%20Orwell List arguments in the query
GET /getBook?name=1984&authorFilter=George%20Orwell Name arguments in the query
GET _/getBook?_<custom implementation> Override the parseQuery method in your ServerSession subclass. See JSDoc. Here's a discussion about different url serializers
GET /book ... Read "GET book" like getBook. Applies to other http verbs also. Additionally "PUT book" will try to call updateBook or setBook cause this sounds more common in programming languages.
POST /getBook {"name": "1984", "authorFilter":"George Orwell"} Name arguments inside JSON body
POST /getBook ["1984", "George Orwell"] List arguments inside JSON body
POST /getBook/1984 "George Orwell" Single JSON primitive
POST /getBook/1984 George Orwell Plain string. For this you must explicitly set the Content-Type header to text/plain
POST /getBook name=1984&authorFilter=George%20Orwell Classic Html <form> with Content-Type = application/x-www-form-urlencoded. Still remember these ? They can be used here as well ;)
POST /getBook/1984 <Any binary data> Binary Data. Your function parameter (i.e. here the 2nd one) must be of type Buffer.

You are free to mix these styles ;) The styles are parsed in the order as listed, so arguments from a lower line in the table will -override named- or -append to listed- ones from above.

Also it's possible to have Readable and Buffers as parameters ...

    async uploadAvatarImage(userName: string, image: Readable) {

    }

...can be called through http like:

Method Url Body Description
POST /uploadAvatarImage/Donald%20Duck <> Binary data directly in the body (TODO)

Content types

To specify what you send and how it should be interpreted, set the Content-Type header to

Auto value conversion

Parameter values will be reasonably auto converted to the actual declared type.

Restfuncs won't try to convert to ambiguous types like string|bool cause that would be too much magic and could cause unwanted behaviour flipping in your app (i.e., someone evil enters 'true' as username and this makes its way to a query param).

Note for the security cautious of you: After all this "wild" parameter collection and auto conversion, the actual call-ready parameters will be security-checked again in a second stage.

Receiving content (json-like result)

To specify what you want to receive in the response, Set the Accept header to

Security

CSRF protection

Tl;dr: In a normal situation (= no basic auth, no client-certs and using the RestfuncsClient), Restfuncs already has a strong CSRF protection by default (corsReadToken, enforced by the RestfuncsClient). For other situations, read the following:

Restfuncs has the following 3 protection levels (weakest to hardest) to protect against CSRF attacks. See list below. You can enforce it by the ServerSessionOptions#csrfProtectionMode setting.
By default/ undefined, the client can decide the protection mode. "wait a minute, how can this be secure ?" See explanation. This way, all sorts of clients can be served. Think of non-browser clients where CSRF does not have relevance, so their devs are not bugged with implementing token fetches.
Explanation: The clients indicate, which csrfProtection mode they want to "play" in a header proactively on every request. Restfuncs will raise an error, if another browser client (or i.e an attacker from another browser tab) wants to play a different mode, at the moment it tries to access the (same) session. Meaning, once the (cookie-) session is created, the protection mode is stored in there. Note: "proactively" means: no header = defaulting to preflight is still allowed, as long as it's consistent.

The above policy (let the clients decide) only covers sessions. So when using client-certificates or basic auth, you must explicitly decide for a setting, and you should use at least set it to corsReadToken when dealing with browser clients.

Here are the modes. ServerSessionOptions#csrfProtectionMode / RestfuncsClient#csrfProtectionMode can be set to:

Notes:

Hardening security for the paranoid

Inline and advanced callbacks

Tl;dr: Restfuncs will (security-) alert, when it can't analyze the type of a callback and tell you what options to adjust. TODO: long version

Performance

To be honest here, this current Restfuncs release's first goal is to be stable and secure. Surely, it will compete with a traditional express handcrafted handlers or usual frameworks, Plus it also has the (web) socket server and there are architectural considerations to avoid round trips and lookups. But real profiling and in-detail optimizations have to be made, to make it competitive to bun, µWebsockets and other high-throughput libraries. Feel free,to benchmark it and contribute to optimizations.

Also in general: You should never talk about performance in theory without having actually profiled your application. I.e. one single simple sql call to a database server (in dimensions of around 100µs on an average cpu) will probably overshadow all the call time of all your communication library. But it's no excuse to not take it sportive ;)... i will focus on this topic later.

Further performance options

Writes to the session fields have some overhead

It costs an additional http roundtrip + 1 websocket roundtrip + (auto.) resend of unprocessed websocket calls. This is to ensure fail-safe commits to the http cookie and to ensure security. So keep that in mind.

Multi server environment

When using a load balancer in front of your servers, you have to configure it for sticky sessions, because the underlying engine.io uses http long polling as a first, failsafe approach. You might try to also change that.

Tips & tricks

Using typescript to automatically trim the output into the desired form

By default, restfuncs trims off all extra properties in the result of your remote methods to match the exact declared typescript type. You can make use of this in combination with these two handy typescript utility types: [Pick<Type, Keys>](using Pick and Omit) and Omit<Type, Keys> Example:

type IUser=  {
  name: string,
  age: number,
  password: string,
}

@remote()
returnsPublicUser(): Pick<IUser, "name" | "age"> { // This will return the user without password
    const user = {name: "Franz", age: 45, password: "geheim!"} // got it from the db somewhere
    return user;
}

@remote()
returnsPublicUser(): Omit<IUser, "password">{  // Also this will return the user without password
   ...
}

or you could also create a new type and go with returnsSafeUser(): SanitizedUser {...}. Etc. etc. you've got all the world of typescript here ;)

Validate stuff on the inside

Now that you've gone all the long way of setting up the build, you have Typia at hand and can use it to validate your objects, i.e. before they get stored the db. Example:

import typia, { tags } from "typia"

type User = {
  name: string & tags.MaxLength<255>
}

if(process.env.NODE_ENV === 'production') { // cause in dev, you usually run without transformed code
  typia.assertEquals<User>(myUser) // Validate myUser before storing it in the db
}
db.store(myUser)

Also you can inspect all your types at runtime

Migration from 2.x

As the 2.x release was announced to be non production-ready, here is how to migrate to the production-ready 3.x version, where those issues were fixed

That's it !

Comparison to other RPC libraries

Comparison table

Contribution

See DEVELOPMENT.md

Places where your help would be needed