search

LionWeb-io / lionweb-repository

Reference implementation of LionWeb repository
Apache License 2.0
2 stars 1 forks source link

Payload too large returned when storing trees #22

Closed ftomassetti closed 5 months ago

ftomassetti commented 5 months ago

I am parsing Java files, producing ASTs and storing them. The operation fails because I get the answer 413 Payload Too Large. I get it on most ASTs. The smallest size causing problems that I noticed is 171461 bytes.

I think we should support much larger files. I had in the past files in the tens of megabytes, that I would expect to be able to store in the repository.

ftomassetti commented 5 months ago

Note that a Java file of 64 bytes, when parsed produce an AST that serialized to JSON takes 14317 bytes. In other words it is 224 times larger than the original file.

So our format is not particularly efficient :D

joswarmer commented 5 months ago

Something like app.use(express.bodyParser({limit: '50mb'})) in the server should solve this.

ftomassetti commented 5 months ago

On my side, I will see if I can enable zipping the payload

joswarmer commented 5 months ago

@ftomassetti Any luck yet?

ftomassetti commented 5 months ago

@joswarmer not yet. I tried some experiments with compression but I have issues activating the compression on the server side. The header Content-Encoding seems not to arrive to the server. I hope to be able to play with this a lit bit more. On the positive side gzip seems to reduce by 20 times the size of the payload