search

nobrainr / morphism

⚡ Type-safe data transformer for JavaScript, TypeScript & Node.js.
https://morphism-playground.now.sh/
MIT License
487 stars 23 forks source link

Thoughts on v2 #47

Open emyann opened 5 years ago

emyann commented 5 years ago

Planning a v2.0.0 for Morphism with those changes:

Please share any ideas or suggestions you would find valuable for a next version :)

davej commented 5 years ago

Awesome, this is a really nice lib.

Probably way outside the scope of what this library is for but it would be cool to be able to output the schema transformations on some sort of diagram. Perhaps an online tool or CLI that outputs to something like d3.js.

emyann commented 5 years ago

@davej This is definitely something I had thought about! I really love the idea, I just didn't know if this could be valuable. Thank you a lot for bringing this on the table, I'll add it to the list :)

reitsma commented 5 years ago

This looks really nice. I am using the JOLT java library in the backend. For the frontend transformations I consider Morphism. I have one specific requirement: The transformation schema must be expressed in JSON. As far as I understand, that can be done with Morphism, my own code will do the (trivial) parse of the JSON before handing the scheme to Morphism.

emyann commented 5 years ago

@reitsma Thank you for considering using Morphism! I don't see any actual blocker for your use case ? If you're able to parse and revive the functions you might have in your schema, it's going to work straight away.

cjancsar commented 5 years ago

Might be nice to have easy hooks to sanitize data during mapping. Can currently do this manually with a function mapper, but might be nice to have a more abstract way.

We are not using typescript at the moment unfortunately.

Think of an integration with sanitizers like validator.js, like express-validator does.

We are using this lib to connect backend services with our internal micro-services, as an ETL-like process. This library is great to map the data, but needs a bit more in the transformation-side.

We are just starting to use it, and I would be happy to help out in the coming months.

emyann commented 5 years ago

@cjancsar This is awesome that you bring this on the table as I'm actually working on the Validation side of Morphism! This is only available on the next branch of this library (npm install morphism@next).

This is not yet ready for production but you can see a sneak peek here:

I added some basic string validators here https://github.com/nobrainr/morphism/blob/next/src/validation/validators/StringValidator.ts like min, max, length, alphanum, regex and planning to add more in the future.

Is it similar to what you had in mind ?

cjancsar commented 5 years ago

@emyann yes! That is exactly what I was thinking.

Do the validators have the ability to change the data, or are they read-only, and only throw errors?

Are you re-creating all your own validators? I know you likely don't want to add an external dependency, but the lib might also get more recognition and wide-spread use if it connected to the already-awesome validator.js which has tons of existing validators.

It also has sanitizers, which coerce the incoming value to another format / clean the data. Would also be nice to see that.

For example:

import { morphism, createSchema, Validation, Sanitization, reporter } from 'morphism';

const source = { name: 'Cassandra', id: '14' };
const schema = createSchema({ 
    id: { 
        fn: (source:any) => source.id, 
        validation: Validation.isInt(),
                sanitization: Sanitization.toInt()
    } 
});

> output = { name: 'Cassandra', id: 14 }

Pretty basic example, but you get the point.

emyann commented 5 years ago

Awesome!

Do the validators have the ability to change the data, or are they read-only, and only throw errors?

Yes actually it does coerce the value to another format and I wanted to give the user the ability to choose the behavior, but I really love the suggestions you shared (Sanitization, integration with validator.js). I'll work on making the validation interface more composable so that it would be easy to plug any other validation / sanitization library rather than taking a dependency on a specific library (if some prefer joi over validator.js it would be possible)

cjancsar commented 5 years ago

@emyann cool! Let me know if there are any low-hanging requirements and I will try and spend some time helping you out.

emyann commented 5 years ago

@cjancsar Yes of course, thank you for offering your help! I'll wrap my head around the integration with 3rd party libraries and prepare the ground for the work. I'll get back to you whenever I start this refactoring.