Compile errors when using TypeScript 2.4.1

OliverJAsh commented 7 years ago

I just tried using the latest fp-ts (currently 0.3.4) with typescript@next (currently 2.5.0-dev.20170622), and I got this error, among others. Do you have any ideas what could be wrong here? Is TypeScript catching a genuine bug?

node_modules/fp-ts/lib/Option.d.ts(20,22): error TS2420: Class 'None<A>' incorrectly implements interface 'FantasyTraversable<"Option", A>'.
  Types of property 'traverse' are incompatible.
    Type '<F extends "Task" | "Option" | "Either" | "io-ts/Type">(applicative: Applicative<F>) => <B, U = a...' is not assignable to type '<F extends "Task" | "Option" | "Either" | "io-ts/Type">(applicative: Applicative<F>) => <B, UF = ...'.
      Type '<B, U = any, V = any>(f: (a: A) => HKT<B, U, V>[F]) => HKT<Option<B>, U, V>[F]' is not assignable to type '<B, UF = any, UT = any, VF = any, VT = any>(f: (a: A) => HKT<B, UF, VF>[F]) => HKT<Option<B>, UF,...'.
        Type 'HKT<Option<{}>, any, any>[F]' is not assignable to type 'HKT<Option<B>, UF, VF>[F]'.
          Type 'Task<Option<{}>> | None<Option<{}>> | Some<Option<{}>> | Left<any, Option<{}>> | Right<any, Optio...' is not assignable to type 'HKT<Option<B>, UF, VF>[F]'.
            Type 'Task<Option<{}>>' is not assignable to type 'HKT<Option<B>, UF, VF>[F]'.
              Type 'Task<Option<{}>>' is not assignable to type 'Task<Option<B>> | None<Option<B>> | Some<Option<B>> | Left<UF, Option<B>> | Right<UF, Option<B>> ...'.
                Type 'Task<Option<{}>>' is not assignable to type 'Task<Option<B>>'.
                  Type 'HKT<Option<{}>, any, any>[F]' is not assignable to type 'Task<Option<B>> | None<Option<B>> | Some<Option<B>> | Left<UF, Option<B>> | Right<UF, Option<B>> ...'.
                    Type 'Task<Option<{}>> | None<Option<{}>> | Some<Option<{}>> | Left<any, Option<{}>> | Right<any, Optio...' is not assignable to type 'Task<Option<B>> | None<Option<B>> | Some<Option<B>> | Left<UF, Option<B>> | Right<UF, Option<B>> ...'.
                      Type 'Task<Option<{}>>' is not assignable to type 'Task<Option<B>> | None<Option<B>> | Some<Option<B>> | Left<UF, Option<B>> | Right<UF, Option<B>> ...'.
                        Type 'Task<Option<{}>>' is not assignable to type 'Task<Option<B>>'.
                          Type 'Option<{}>' is not assignable to type 'Option<B>'.
                            Type 'None<{}>' is not assignable to type 'Option<B>'.
                              Type 'None<{}>' is not assignable to type 'None<B>'.
                                Type '{}' is not assignable to type 'B'.
                                  Type 'HKT<Option<{}>, any, any>[F]' is not assignable to type 'Type<Option<B>>'.
                                    Type 'Task<Option<{}>> | None<Option<{}>> | Some<Option<{}>> | Left<any, Option<{}>> | Right<any, Optio...' is not assignable to type 'Type<Option<B>>'.
                                      Type 'Task<Option<{}>>' is not assignable to type 'Type<Option<B>>'.
                                        Property 'name' is missing in type 'Task<Option<{}>>'.

gcanti commented 7 years ago

Is TypeScript catching a genuine bug?

I don't thinks so. It rather seems that v2.5 is handling type parameters in a different way. Needs more investingations though

OliverJAsh commented 7 years ago

I think I also saw this with 2.4

gcanti commented 7 years ago

@OliverJAsh just tried 2.4.1, the build is broken

sledorze commented 7 years ago

@gcanti, from the Typescript team answer https://github.com/Microsoft/TypeScript/issues/16806 it looks like the awesome approach is now compromised. is it a show stopper or is there a work around (without loosing on type safety)?

gcanti commented 7 years ago

it looks like the awesome approach is now compromised

@sledorze Maybe. After many hours of trials I'm still not sure what's exactly wrong

sledorze commented 7 years ago

@gcanti isn't that something related to co / contravariance in functions assignment ?

gcanti commented 7 years ago

@sledorze yeah, but it's not clear to me which rules ts is using to check co / contravariance. Based on my observations so far I think is still a mix of sound and unsound behavior, so even if I find a fix (improbable) would be brittle and every future release could break fp-ts again (the main culprit here seems how I'm faking HKTs).

So the only sensible solution seems

doing a major refactoring
switching back to the first technique for faking HKTs
start from the ground up and add step by step the most important type classes and data structures making sure, as far as possible, that will be future proof
pushing the new version on a branch with lib commited in
check how many breaks we get in our codebases
then switch to PureScript (...:smile:)

gcanti commented 7 years ago

switching back to the first technique for faking HKTs

Note: this is just a speculation, I have to write a proof of concept to make sure there are real benefits with 2.4.1

gcanti commented 7 years ago

So, here's the POC https://github.com/gcanti/fp-ts-poc

HKTs are faked with the old technique (see https://medium.com/@gcanti/higher-kinded-types-in-typescript-static-and-fantasy-land-d41c361d0dbe)

TypeScript 2.4.1 seems happy with that

However the old technique, as we know, leads to bad inference. To fix that I added these overloadings

(I know, is pretty ugly, but seems to work...)

Demo

import { sequence } from 'fp-ts-poc/lib/Traversable'
import * as option from 'fp-ts-poc/lib/Option'
import * as array from 'fp-ts-poc/lib/Array'
import { lift } from 'fp-ts-poc/lib/Functor'
import * as either from 'fp-ts-poc/lib/Either'

// x :: option.Option<number[]>
export const x = sequence(option, array)([option.some(1)])

const length = (s: string): number => s.length
const double = (n: number): number => n * 2

// maybeLength :: (fa: option.Option<string>) => option.Option<number>
const maybeLength = lift(option, length)

option.map(double, maybeLength(option.some('hello')))

// y: either.Either<string, number[]>
export const y = sequence(either, array)([either.right<string, number>(1)])

What do you think?

KiaraGrouwstra commented 7 years ago

To fix that I added these overloadings

Curious, would you have examples of before / after?

Would you also have an example of how the inference went wrong?

gcanti commented 7 years ago

Without overloadings

// x :: HKT<"Option", HKT<"Array", {}>>
export const x = sequence(option, array)([option.some(1)])

const length = (s: string): number => s.length
const double = (n: number): number => n * 2

// maybeLength :: (fa: HKT<"Option", string>) => HKT<"Option", number>
const maybeLength = lift(option, length)

option.map(double, maybeLength(option.some('hello'))) // error: Argument of type 'HKT<"Option", number>' is not assignable to parameter of type 'Option<number>'

// y :: HKT<"Either", HKT<"Array", {}>>
export const y = sequence(either, array)([either.right<string, number>(1)])

KiaraGrouwstra commented 7 years ago

I feel like the API actually got a bit prettier this way without all the quote noise. 😃

sledorze commented 7 years ago

@gcanti that's a lot of specialisation but at least it works, that's better than the total breakage! :)

DylanRJohnston commented 7 years ago

then switch to PureScript (...😄)

If only I could convince my product manager

sledorze commented 7 years ago

Purescript is very, very, very nice, BUT you need to write a lot of externs to use some js libs (and they often don't map that well to PS). Not even mentioning the nodeJs story..

DylanRJohnston commented 7 years ago

I find myself pretty frequently writing typing definitions for Typescript too

gcanti commented 7 years ago

The good news is that with the old technique and the improved type inference of 2.4.1, implementations are way better. For example

before

liftA4<F extends HKTS, A, B, C, D, E>(
  apply: Apply<F>,
  f: Curried4<A, B, C, D, E>
): <U = any, V = any>(
  fa: HKT<A, U, V>[F],
  fb: HKT<B, U, V>[F],
  fc: HKT<C, U, V>[F],
  fd: HKT<D, U, V>[F]
) => HKT<E, U, V>[F] {
  return (fa: HKT<A>[F], fb: HKT<B>[F], fc: HKT<C>[F], fd: HKT<D>[F]) =>
    apply.ap<D, E>(
      apply.ap<C, (d: D) => E>(apply.ap<B, Curried2<C, D, E>>(apply.map<A, Curried3<B, C, D, E>>(f, fa), fb), fc),
      fd
    )
}

https://github.com/gcanti/fp-ts/blob/9ccd624d257a2ba0083237b2473542679c324cb0/src/Apply.ts#L58

after

liftA4<F, A, B, C, D, E>(
    apply: Apply<F>,
    f: Curried4<A, B, C, D, E>
  ): (fa: HKT<F, A>, fb: HKT<F, B>, fc: HKT<F, C>, fd: HKT<F, D>) => HKT<F, E> {
    // awesome type inference here
    return (fa, fb, fc, fd) => apply.ap(apply.ap(apply.ap(apply.map(f, fa), fb), fc), fd) 
  }

https://github.com/gcanti/fp-ts-poc/blob/b59a9f4f776baf63cee0d090b7e970a01b81a06e/src/Apply.ts#L57

KiaraGrouwstra commented 7 years ago

Just wait until we can make things generic and ditch the duplication. 😃

gcanti commented 7 years ago

Hi folks, I need your help: this is the list of breaking changes in fp-ts-poc, could you please tell me what must be re-implemented in order to check the new implementation in your projects? Thanks

Breaking changes

Applicative.getCompositionApplicative
Dictionary
EitherT
Filterable
Foldable.getCompositionFoldable
Free
FreeAp
Functor.getCompositionFunctor
Id
~~Identity~~ (implemented)
IO (implemented)
IxMonad
Mealy
NaturalTransformation
~~NonEmptyArray~~ (implemented)
OptionT
Profunctor
Reader
ReaderT
~~Semigroupoid~~ (implemented)
State
StateT
Store
~~Task~~ (implemented)
These
Traced
Tuple
~~Validation~~ (implemented)
Witherable
Writer

sledorze commented 7 years ago

@gcanti we're using it with io-ts, would that one need to be updated to? Otherwise, we're mainly using those from the remaining:

Applicative.getCompositionApplicative
EitherT
Foldable.getCompositionFoldable
Functor.getCompositionFunctor
Id
Identity
OptionT
Semigroupoid
Tuple

Also not sure that all our libs are 2.4.1 compliant (I've seen angular can break with it). We'll test as much as we can! :)

gcanti commented 7 years ago

we're using it with io-ts, would that one need to be updated to?

just bump the fp-ts dependency (Either should be backward compatible)

Id, Tuple

I'm not sure they can be ported, how are you using them?

For what concerns the rest

Should be ok

Identity
Semigroupoid

Under investigation...

Applicative.getCompositionApplicative
EitherT
Foldable.getCompositionFoldable
Functor.getCompositionFunctor
OptionT

sledorze commented 7 years ago

@gcanti we use Id in some tests of a generic implementation, in prod it uses the Task monad whereas in the tests it uses Id. As for Tuple, I would have to check again (away from computer ATM)

gcanti commented 7 years ago

@sledorze thanks, hopefully the usages of Id may be replaced by Identity without much hassle. The problem with tuples is that is no more possible to see them as HKT without wrapping. We could keep the Ord / Monoid instances and some utility functions though.

A note of encouragement. For what I've seen so far, this change is worth it: the simpler technique for HKTs combined with the improved type inference of 2.4.1 is very nice! I'm getting rid of tons of boilerplate type annotations in the implementations and the error messages are way better (no more long silly lists of unions)

As a cons, I'm writing hundreds of lines of overloadings..

SimonMeskens commented 7 years ago

What's the migration path from the POC to the next version of fp-ts? Can I start using the POC now and then replace it with the next version of fp-ts eventually? Can we get a little documentation on how to do HKT with the POC?

gcanti commented 7 years ago

What's the migration path from the POC to the next version of fp-ts?

@SimonMeskens The POC is supposed to become the next version of fp-ts

Can I start using the POC now and then replace it with the next version of fp-ts eventually?

Yes, but keep in mind that's a bit risky, I'm still fleshing out some important details

Can we get a little documentation on how to do HKT with the POC?

HKTs are faked with this technique https://medium.com/@gcanti/higher-kinded-types-in-typescript-static-and-fantasy-land-d41c361d0dbe

The idea is based on the paper Lightweight higher-kinded polymorphism.

gcanti commented 7 years ago

@sledorze I have a POC of:

Applicative.getCompositionApplicative EitherT Foldable.getCompositionFoldable Functor.getCompositionFunctor OptionT

For example EitherT https://github.com/gcanti/fp-ts-poc/blob/452a7765e08185bb9fb14fdd0c50703b0df9ebb7/src/EitherT.ts Usage: https://github.com/gcanti/fp-ts-poc/blob/452a7765e08185bb9fb14fdd0c50703b0df9ebb7/examples/TaskEither.ts

gcanti commented 7 years ago

Update. After 8 days of frantic work I almost finished the rewrite, here's the list of current breaking changes

Tuple (wrapped)
Dictionary (wrapped, renamed to StrMap)
changed
- Applicative.getCompositionApplicative
- Foldable.getCompositionFoldable
- Functor.getCompositionFunctor
- Traversable.getCompositionTraversable
- Free (usage)
- NaturalTransformation
- ReaderT
- StateT
removed (temporarily or because the porting is not possible)
- Id (not possible)
- Traced
- IxMonad
- Mealy
- FreeAp

Any comment or suggestion is welcomed

OliverJAsh commented 7 years ago

@gcanti Your work is really appreciated. Where can I donate to you? :-)

gcanti commented 7 years ago

@OliverJAsh Thanks. Well if I can dare I'd say donate some of your time (occasionally of course). I'd love to accept any PR involving the documentation (for this or other related libraries) which is admittely pretty poor and I feel bad about it.

SimonMeskens commented 7 years ago

Stellar job! If I end up using this library for a larger project that lets me get more acquainted with it, I'll be sure to PR some documentation.

sledorze commented 7 years ago

@gcanti thanks for the hard work! It looks like it is even now more readable :) I've not be able to test it on our code base, because it fails for other lib incompatibilities with TS 2.4.1 (and missing time to tweak those). I may give a try when coming back from holidays (end of July).

gcanti commented 7 years ago

closed this in 03ee0fa33b248383ae566d0a61d52049f484108b

gcanti commented 7 years ago

FYI just released v0.4.0 in the next channel if you want to try it out

npm install fp-ts@next

OliverJAsh commented 7 years ago

@gcanti I just tried updating to io-ts next and fp-ts next in my small project decode-ts. Unfortunately I seem to be running into memory issues again (FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory).

You can try it for yourself by checking out the repo, running yarn add fp-ts@next io-ts@next and then tsc.

It's a small project so it should be relatively straightforward to narrow down the memory issue and file an issue against TypeScript. Until I can workaround the memory issue I won't be able to upgrade my other project twitter-api-ts, which depends on decode-ts.

DylanRJohnston commented 7 years ago

Also running into the same issues

OliverJAsh commented 7 years ago

It turned out to be because I had a dependency with its own versions of fp-ts and io-ts that differed from the ones in the root package. It's compiling now, and I had to make 0 changes.

DylanRJohnston commented 7 years ago

Can't seem to find the dependency version mismatch, but I'll keep looking.

SimonMeskens commented 7 years ago

the out of memory error has nothing to do with fp-ts, but does affect a large number of other libraries. Immutable.js is broken with 2.4.1, as is RxJS and many, many others. For now, the easiest fix is to set "skipLibCheck" to true in your tsconfig. This skips type-checking for library files. I'm not sure what the ramifications of not typechecking libraries are.

gcanti commented 7 years ago

@OliverJAsh great.

@DylanRJohnston @SimonMeskens I also experimented some failure due to package duplication in dependencies as @OliverJAsh said but it's true that with ts 2.4.1 many libraries are broken so it could be both I guess, depending on the project deps

Though I didn't extensive testing, fp-ts@0.4.0 should be consumable by previous versions of TypeScript other than 2.4.1 (in particular ts 2.4.0)

gcanti commented 7 years ago

@sledorze after some thought, while fp-ts@0.4.0 is developed with ts 2.4.1, since it doesn't use default type parameters anymore in its implementation, should be compatible even with ts 2.2.2

p.s. have a nice holidays

gcanti / fp-ts

Compile errors when using TypeScript 2.4.1 #138