catamphetamine / read-excel-file

Read *.xlsx files in a browser or Node.js. Parse to JSON with a strict schema.
https://catamphetamine.gitlab.io/read-excel-file/
MIT License
301 stars 52 forks source link

(Typescript) Add typings #71

Closed ChamNouki closed 3 years ago

ChamNouki commented 4 years ago

Hello Nikolay,

This library seems very very cool to use but unfortunately I can't because it lacks typings for typescript based project with unauthorized javascript import.

Never mind I start to create typings to be able to use it xD. Have you already considered adding typings to your project ? Do you want some help for ? I can PR my typings if you are interested into.

catamphetamine commented 4 years ago

Have you already considered adding typings to your project ?

Hi. I'm not a TypeScript user myself, so I haven't.

I can PR my typings if you are interested into.

Sure. I'll merge whatever you submit, but you'll have to check that they work.

ChamNouki commented 4 years ago

Hello,

I'll finally not create PR has I will finally not use your library due to performance issue over non small sheets. Any way I give you the typings I created. It's not complete but it's usable.

(to put in an index.d.ts)

declare module 'read-excel-file/node' {
  import { PathLike } from 'fs';
  import { Stream } from 'stream';

  export type SchemaPropertyType =
    | 'Integer'
    | 'URL'
    | 'Email'
    | typeof Date
    | typeof Number
    | typeof Boolean;

  export interface FinalSchemaProperty {
    prop: string;
    type: SchemaPropertyType;
    required?: boolean;
    parse?<T>(value: string): T;
    validate?<T>(value: T): void;
  }

  export interface MiddleSchemaProperty {
    prop: string;
    type: Record<string, SchemaProperty>;
  }

  export type SchemaProperty = FinalSchemaProperty | MiddleSchemaProperty;

  export interface SheetParsingOptions {
    sheet?: number | string;
    properties?: any;
    rowMap?: any;
    isColumnOriented?: boolean;
    transformData?(data: any): any;
  }

  export interface SheetConvertOptions extends SheetParsingOptions {
    schema: Record<string, SchemaProperty>;
  }

  export interface SheetNamesOptions {
    getSheets: boolean;
  }

  export type SheetOptions = SheetParsingOptions | SheetNamesOptions;

  export interface ParsedResult {
    rows: any[];
    errors: any[];
  }

  function readXlsxFile(
    input: Stream | PathLike,
    options?: SheetOptions
  ): Promise<ParsedResult>;
  function readXlsxFile(
    input: Stream | PathLike,
    options: SheetConvertOptions
  ): Promise<ParsedResult>;

  export default readXlsxFile;
}
catamphetamine commented 4 years ago

Ok. I'll reopen this issue so that other people who would want TypeScript could base their PRs on your code. I'll also add the note in the readme that the library is better suited for smaller files.

pavanagrawal123 commented 4 years ago

Hi @catamphetamine, thanks for working on this. Can we merge this into master, it would be helpful for TS users. I can submit a PR if wanted.

catamphetamine commented 4 years ago

@pavanagrawal123

Hi @catamphetamine, thanks for working on this. Can we merge this into master, it would be helpful for TS users. I can submit a PR if wanted.

Sure, TypeScript pull requests would be accepted, if there is another man reviewing them.

tgmarinho commented 4 years ago

Hello,

I'll finally not create PR has I will finally not use your library due to performance issue over non small sheets. Any way I give you the typings I created. It's not complete but it's usable.

(to put in an index.d.ts)

declare module 'read-excel-file/node' {
  import { PathLike } from 'fs';
  import { Stream } from 'stream';

  export type SchemaPropertyType =
    | 'Integer'
    | 'URL'
    | 'Email'
    | typeof Date
    | typeof Number
    | typeof Boolean;

  export interface FinalSchemaProperty {
    prop: string;
    type: SchemaPropertyType;
    required?: boolean;
    parse?<T>(value: string): T;
    validate?<T>(value: T): void;
  }

  export interface MiddleSchemaProperty {
    prop: string;
    type: Record<string, SchemaProperty>;
  }

  export type SchemaProperty = FinalSchemaProperty | MiddleSchemaProperty;

  export interface SheetParsingOptions {
    sheet?: number | string;
    properties?: any;
    rowMap?: any;
    isColumnOriented?: boolean;
    transformData?(data: any): any;
  }

  export interface SheetConvertOptions extends SheetParsingOptions {
    schema: Record<string, SchemaProperty>;
  }

  export interface SheetNamesOptions {
    getSheets: boolean;
  }

  export type SheetOptions = SheetParsingOptions | SheetNamesOptions;

  export interface ParsedResult {
    rows: any[];
    errors: any[];
  }

  function readXlsxFile(
    input: Stream | PathLike,
    options?: SheetOptions
  ): Promise<ParsedResult>;
  function readXlsxFile(
    input: Stream | PathLike,
    options: SheetConvertOptions
  ): Promise<ParsedResult>;

  export default readXlsxFile;
}

Thanks, I put this in my project and it works.

Ge11ert commented 4 years ago

Hey, big deal with typings, but I'm not sure about

interface ParsedResult {
    rows: any[];
    errors: any[];
  }

Readme says, the main script returns rows from promise, where:

rows is an array of rows, each row being an array of cells.

So actually what we have here is rows: any[][].

And offered typings declare that return value is object with fileds rows and errors, but that's not true.

Am I right? If not, fix me, pls

catamphetamine commented 4 years ago

Rows is an array of arrays when no schema is passed. Otherwise, it’s an array of objects.

On Sun, 7 Jun 2020 at 11:16, Artyom notifications@github.com wrote:

Hey, big deal with typings, but I'm not sure about

interface ParsedResult { rows: any[]; errors: any[]; }

Readme says, the main script returns rows from promise, where:

rows is an array of rows, each row being an array of cells.

So actually what we have here is rows: any[][].

And offered typings declare that return value is object with fileds rows and errors, but that's not true.

Am I right? If not, fix me, pls

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/catamphetamine/read-excel-file/issues/71#issuecomment-640175956, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADUP33FM5ME7FG7E4Z3UV3RVNEEXANCNFSM4K3PCCCQ .

Ge11ert commented 4 years ago

Rows is an array of arrays when no schema is passed. Otherwise, it’s an array of objects.

Ah, my bad, skipped JSON-part in Readme. Then there may be a union type, something like

  export interface JSONParsedResult {
    rows: any[];
    errors: any[];
  }

  export type ParsedResult = any[][] | JSONParsedResult;

Although I don't know is it a good idea to concat type and interface

ChamNouki commented 4 years ago

What i understand is that only the rows field type change if schema is passed (or not). Errors is always present. So the rows is either an array of array of cells, or an array of objects. But objects type depends of schema so it's not really possible to set type for theme. It's the same for cells, you don't have the type information. And cells and objects are not always the same inside one array !

Conclusion rows could be an array of array of any => Array<Array<any>> == Array<any[]> == any[][] or an array of any => any[]. Array of any (Array<any>) is included in any so finally : Array<Array<any>> => Array<any> == any[] => both are array of any ^_^.

catamphetamine commented 4 years ago

I've received a similar request today in another repo, so I decided to review the typings. @ChamNouki Your typings seem useful. I've based my variation on your typings.

The typings drafts are in the repo:

The instructions for testing those typings would be:

Also, here's a copy-pasted "universal" version of those typings (one could place the following definitions, wrapped with declare module read-excel-file { ... } or declare module read-excel-file/node, in some definition file *.d.ts in your project and see if those definitions work):

// import { PathLike } from 'fs'
// import { Stream } from 'stream'

type BasicType =
    | string
    | number
    | boolean
    | typeof Date
    | 'Integer'
    | 'URL'
    | 'Email'

interface SchemaEntryBasic {
    prop: string;
    type: BasicType;
    oneOf?<T>: T[];
    required?: boolean;
    validate?<T>(value: T): void;
}

interface SchemaEntryParsed {
    prop: string;
    parse<T>: (value: string) => T?;
    oneOf?<T>: T[];
    required?: boolean;
    validate?<T>(value: T): void;
}

// Implementing recursive types in TypeScript:
// https://dev.to/busypeoples/notes-on-typescript-recursive-types-and-immutability-5ck1
interface SchemaEntryRecursive {
    prop: string;
    type: Record<string, SchemaEntry>;
    required?: boolean;
}

type SchemaEntry = SchemaEntryBasic | SchemaEntryParsed | SchemaEntryRecursive

export type Schema = Record<string, SchemaEntry>

export interface Error {
    error: string;
    row: number;
    column: number;
    value?: any;
    type?: SchemaEntry;
};

type Cell = string | number | boolean | typeof Date
export type Row = Cell[]

type InputBrowser = File
type InputNode = any // Stream | PathLike // (string|Stream|Buffer)
export type Input = InputBrowser | InputNode

export interface ParsedObjectsResult {
    rows: object[];
    errors: Error[];
}

export interface ParseWithSchemaOptions {
    schema: Schema;
    transformData?: (rows: Row[]) => Row[];
    sheet?: number | string;
}

export interface ParseWithoutSchemaOptions {
    sheet?: number | string;
}

function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>;
function readXlsxFile(input: Input, options?: ParseWithoutSchemaOptions) : Promise<Row[]>;

export default readXlsxFile;

@ChamNouki One thing I'm not sure about though is this declare module xxx {} thing: should it be present or should it be omitted? For example, in one of my other libraries we have index.d.ts without any declare module stuff and it seems to work. So should declare module be added?

Also, do all interfaces and types have to be exported? Or only some of them?

In the meantime, the official not-tested index.d.ts is (will be updated in this comment if any issues are found):

@ChamNouki Also, it says type InputNode = any because I don't know if Stream | PathLike or string|Stream|Buffer would work in a non-Node.js project. They say that it would require installing some kind of an additional @type/node typings package manually which wouldn't be an appropriate solution for "novices". And I didn't check if type InputBrowser = File requires any imports in order to work, or whether File some kind of a "global" type.

ChamNouki commented 4 years ago

Hi @catamphetamine ,

The declare module xxx {} in my definition file is because my d.ts file was in my project and not in the read-excel-file node_module folder. So I have to tell to Typescript that those definitions are the ones for the read-excel-file library. If you provide the library type definitions in your project, no need for that ;)

About exporting interfaces and types, it depends : You have to export types that will be used by the library users. If some types are just internal definition, no need to export them.

You're wright Stream, PathLike and Buffer are node types and wont be available without installing @type/node. However if you use those objects in your code, your library will probably not work on browsers or may bring the node code in the generated bundle (didn't test) ? Types you can import from @types/node are for objects that are only available on Node environment. Wich is not the case for File type which is some kind of "global" type available in browsers too.

DPros commented 3 years ago

there's a index.d.ts.test file in the package read-excel-file@5.1.0. after renaming it to index.d.ts typings are working great. Can't it be properly named initially? hoping it can be resolved soon, thx!

catamphetamine commented 3 years ago

Added TypeScript definitions in read-excel-file@5.2.13.

sinn1 commented 3 years ago

@catamphetamine getting typescript errors:


19 function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>;
   ~~~~~~~~

node_modules/read-excel-file/types.d.ts:10:4 - error TS2749: 'Integer' refers to a value, but is being used as a type here. Did you mean 'typeof Integer'?

10  | Integer
      ~~~~~~~

node_modules/read-excel-file/types.d.ts:12:4 - error TS2749: 'Email' refers to a value, but is being used as a type here. Did you mean 'typeof Email'?

12  | Email;
      ~~~~~

node_modules/read-excel-file/types.d.ts:14:40 - error TS8020: JSDoc types can only be used inside documentation comments.

14 export type Type = <T>(value: Cell) => T?;
                                          ~~

node_modules/read-excel-file/types.d.ts:26:26 - error TS8020: JSDoc types can only be used inside documentation comments.

26  parse: (value: Cell) => T?;
                            ~~

node_modules/read-excel-file/types.d.ts:50:2 - error TS1036: Statements are not allowed in ambient contexts.

50 };
soufianerafik-jd commented 3 years ago

@catamphetamine getting typescript errors:


19 function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>;
   ~~~~~~~~

node_modules/read-excel-file/types.d.ts:10:4 - error TS2749: 'Integer' refers to a value, but is being used as a type here. Did you mean 'typeof Integer'?

10  | Integer
      ~~~~~~~

node_modules/read-excel-file/types.d.ts:12:4 - error TS2749: 'Email' refers to a value, but is being used as a type here. Did you mean 'typeof Email'?

12  | Email;
      ~~~~~

node_modules/read-excel-file/types.d.ts:14:40 - error TS8020: JSDoc types can only be used inside documentation comments.

14 export type Type = <T>(value: Cell) => T?;
                                          ~~

node_modules/read-excel-file/types.d.ts:26:26 - error TS8020: JSDoc types can only be used inside documentation comments.

26  parse: (value: Cell) => T?;
                            ~~

node_modules/read-excel-file/types.d.ts:50:2 - error TS1036: Statements are not allowed in ambient contexts.

50 };

I just upgraded to the 5.2.13 and still getting these errors:

 Error: node_modules/read-excel-file/index.d.ts:13:1 - error TS1046: Top-level declarations in .d.ts files must start with either a 'declare' or 'export' modifier.

 13 function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>;
    ~~~~~~~~

 Error: node_modules/read-excel-file/types.d.ts:13:4 - error TS2749: 'Integer' refers to a value, but is being used as a type here. Did you mean 'typeof Integer'? Integer
      ~~~~~~~

Error: node_modules/read-excel-file/types.d.ts:15:4 - error TS2749: 'Email' refers to a value, but is being used as a type here. Did you mean 'typeof Email'?

 15  | Email
       ~~~~~ 
 Error: node_modules/read-excel-file/types.d.ts:22:11 - error TS1005: '(' expected.

 22  oneOf?<T>: T[];

 Error: node_modules/read-excel-file/types.d.ts:29:10 - error TS1005: '(' expected.

 29  parse<T>: (value: Cell) => T?;

 Error: node_modules/read-excel-file/types.d.ts:30:11 - error TS1005: '(' expected.

 30  oneOf?<T>: T[];
catamphetamine commented 3 years ago

@sinn1 published read-excel-file@5.2.14

catamphetamine commented 3 years ago

@soufianerafik-jd parse<T>: (value: Cell) => T? is from some old version. You didn't update.

soufianerafik-jd commented 3 years ago

@soufianerafik-jd parse<T>: (value: Cell) => T? is from some old version. You didn't update.

Yes, the issue I reported was on the read-excel-file@5.2.13. I just bumped the version to read-excel-file@5.2.14 and it works.

Thank you so much for your help on this. 😄

sigcito commented 3 years ago

Getting this error.

ERROR in ../node_modules/read-excel-file/index.d.ts:13:1 - error TS1046: A 'declare' modifier is required for a top level declaration in a .d.ts file.

13 function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise;

sigcito commented 3 years ago

Solved with adding export to these functions (using this ver. read-excel-file@5.2.15). @catamphetamine

export function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>; export function readXlsxFile(input: Input, options: ParseWithMapOptions) : Promise<ParsedObjectsResult>; export function readXlsxFile(input: Input, options?: ParseWithoutSchemaOptions) : Promise<Row[]>;

Instead of

function readXlsxFile(input: Input, options: ParseWithSchemaOptions) : Promise<ParsedObjectsResult>; function readXlsxFile(input: Input, options: ParseWithMapOptions) : Promise<ParsedObjectsResult>; function readXlsxFile(input: Input, options?: ParseWithoutSchemaOptions) : Promise<Row[]>;

catamphetamine commented 3 years ago

@SafetyOwl Thx, published read-excel-file@5.2.17.