search

textileio / js-textile

Textile's JavaScript Libs. Home of ThreadDB, Buckets, and more. Available on npm as `@textile/hub`.
https://textileio.github.io/js-textile/docs
MIT License
84 stars 19 forks source link

Google GRPC dependencies uses eval that blocks using the SDK on Cloudflare Workers #1166

Open n1ru4l opened 3 years ago

n1ru4l commented 3 years ago

Has anyone successfully used the @textile/threads-client sdk for connecting to ThreadDB from within a cloudflare worker? I am getting the following error from the wrangler cli:

EvalError: Code generation from strings disallowed for this context
    at Function (<anonymous>)
    at worker.js:114:246327
    at createCommonjsModule (worker.js:114:878)
    at Module../node_modules/@textile/threads-client/dist/esm/index.js (worker.js:114:246268)
    at __webpack_require__ (worker.js:663:367)
    at Module../node_modules/@textile/threaddb/dist/esm/index.js (worker.js:60:776)
    at __webpack_require__ (worker.js:663:367)
    at Object../lib/schema-store.ts (worker.js:582:2297)
    at __webpack_require__ (worker.js:663:367)
    at Object../pages/api/schema/[schemaId].ts (worker.js:585:2352)

According to https://github.com/cloudflare/wrangler/issues/1268 cloudflare workers does not allow running eval. I quickly checked the node_modules/@textile/threads-client/dist/esm/index.js and it seems to be full of eval usages :confused: (I am using the @textile/hub package)

yarn why @textile\threads-client
yarn why v1.22.10
warning ..\..\package.json: No license field
[1/4] Why do we have the module "@textile\\threads-client"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "@textile/threads-client@2.2.0"
info Reasons this module exists
   - "@textile#hub#@textile#hub-threads-client" depends on it
   - Hoisted from "@textile#hub#@textile#hub-threads-client#@textile#threads-client"
Now I tried favoring the cjs/browser bundles over esm via webpack config and I am receiving the following:
Uncaught
EvalError: Code generation from strings disallowed for this context
    at Function (<anonymous>)
    at Object../node_modules/@textile/threads-client-grpc/threads_pb.js (worker.js:148:169)
    at __webpack_require__ (worker.js:1080:254)
    at Object../node_modules/@textile/threads-client/dist/cjs/index.js (worker.js:154:1660)
    at __webpack_require__ (worker.js:1080:254)
    at Object../node_modules/@textile/hub-threads-client/dist/cjs/index.js (worker.js:112:151)
    at __webpack_require__ (worker.js:1080:254)
    at Object../node_modules/@textile/grpc-authentication/dist/cjs/index.js (worker.js:76:684)
    at __webpack_require__ (worker.js:1080:254)
    at Object../node_modules/@textile/buckets/dist/cjs/buckets.js (worker.js:49:567)
Uncaught (in response)
Error: EvalError: Code generation from strings disallowed for this context
Uncaught (in response)
Error: EvalError: Code generation from strings disallowed for this context

I assume the issue here is the google grpc usage :confused: (edited)

Tarnadas commented 3 years ago

I ran into the same issue. Is this related to protocolbuffers/protobuf-javascript#25?

Would it be possible to replace the grpc library with another one, like this pure JS implementation: https://www.npmjs.com/package/@grpc/grpc-js (in a reasonable amount of time)?

I haven't tested any other serverless functions provider. Does anyone have experiences with other providers?

Tarnadas commented 3 years ago

I made a little more research and it seems like Google Closure Compiler generates the evals when building the google-protobuf dependency. I'm fiddling around with it right now and see, if there is a way to avoid them, but it's not the most pleasant experience.

I've also been asking on the CF Workers Discord server whether there is any way to upload such functions that involve unsafe eval, but I got an answer that it's not possible.

carsonfarmer commented 3 years ago

Thanks for digging into this @Tarnadas. I did some research as well, and the closest thing I've come up with is actually just trying to use sed, awk, or similar to remove those lines from the built code. Not ideal, but possibly doable?

Tarnadas commented 3 years ago

I managed to fix the issue with some nasty Webpack transformations, which basically removed all eval and Function usages, but I stumbled across the next issue using Websockets on CF Workers. I don’t think this is easy to solve either, which is why I gave up.

Do you know if it is possible to use ThreadDB in another serverless environment?

carsonfarmer commented 3 years ago

Unfortunately, we haven't tested this elsewhere. Maybe @andrewxhill has some thoughts, but I think this usage just hasn't really been on our radar?

andrewxhill commented 3 years ago

Right, I believe we did some tests with the pure js grpc library in the past. I'm trying to recall what the blockers were, it may have been around the unary type requests?

carsonfarmer commented 3 years ago

The issue was with the bidi calls. Unfortunately, I think this is still an issue. @Tarnadas, if you are interested, there is the possibility that you might be able to do this via some other means... perhaps if you only need a limited set of APIs, the pure gRPC route is possible?

good-lly commented 1 week ago

@n1ru4l @Tarnadas great find! You saved me a few days of work! I stumble on almost the same issue w Avro-js / Avsc - CF CSP is blocking some generated eval-like code. Also hackable by nasty transformation but 100% would avoid it ...