Closed abdallahkadour closed 1 year ago
Every time I have to touch anything related to NativeScript, nothing ever works the way it did a month or two ago. After trying for over an hour, I couldn't get anything to build this time. :(
Can you show me the output of console.log(object)
, console.log(encoded)
, and console.log(JSON.stringify(decoded))
as well, please?
If we can't figure this out remotely, I'll probably need some help to create an environment that compiles.
Of course
CONSOLE LOG: the object {
"x": "val_x",
"y": "val_y"
}
CONSOLE LOG: the encoded object: �axeval_xayeval_y
CONSOLE LOG: the decoded object: {"¢axeval_xayeval_y": "¢axeval_xayeval_y"}
CONSOLE LOG: the JSON string of decoded: {"¢axeval_xayeval_y":"¢axeval_xayeval_y"}
I want to mention that I am only using the decode function. It fails when decoding a cbor which I can parse problemless using other tools.
It looks like enocde() is return a string, which i quite odd. I wonder what buffer
implementation you're getting. What is console.log(encoded.toString('hex'))
? How about console.log(encoded[0])
?
CONSOLE LOG: hex a261786576616c5f7861796576616c5f79
CONSOLE LOG: encoded[0] 162
ok, so encode is working correctly, but that's a really odd default toString routine for your buffer class. Does require.resolve("buffer/")
return something like ".../node_modules/buffer/index.js"?
Can you try console.log(await cbor.diagnose(encoded))
.
Oh, I see. That is the default toString for the external buffer
module.
Wait... are you sure CONSOLE LOG: the decoded object: {"¢axeval_xayeval_y": "¢axeval_xayeval_y"}
is actually the output of cbor.decode? ¢
is U+00A2, and a
is U+0061, which matches some sort of decoding of encoded as a string, rather than as CBOR.
Yes I am sure this is my code:
import process from 'process'
global.process = process
import prereqs from 'cbor-rn-prereqs'
import cbor from 'cbor'
let object = {
x: 'val_x',
y: 'val_y'
}
let encoded = cbor.encode(object)
let decoded = cbor.decode(encoded)
console.log('hex', encoded.toString('hex'))
console.log('encoded[0]', encoded[0])
console.log('the object', object)
console.log('the encoded object', encoded)
console.log('the decoded object', decoded)
console.log('JSON string', JSON.stringify(decoded))
the diagnose log: CONSOLE LOG: diagnose {"¢axeval_xayeval_y": "¢axeval_xayeval_y", "¢axeval_xayeval_y": "¢axeval_xayeval_y"}
At this point, I probably need to play with it myself in a debugger. Could you create a small github repo that has your code in a working environment for me, please?
Of course: debug-repo
OK, I've found the problem and will think about a work-around. in NS, TextDecoder
exists now, but it doesn't deal with slices of Buffers correctly, instead decoding the whole original Buffer.
To see the problem without 3rd party bits:
const td = new TextDecoder('utf8', {fatal: true, ignoreBOM: true})
const buf = new Uint8Array([97, 98, 99, 100]);
const s = new Uint8Array(buf.buffer, buf.byteOffset + 1, 1);
const dec = td.decode(s)
console.log('utf8', buf, s, dec)
// utf8 97,98,99,100 98 abcd
(expected "b" instead of "abcd")
Here is the error: https://github.com/NativeScript/NativeScript/blob/17c85107ba84953630b0471c1f6f3d68f6d59f76/packages/core/text/text-common.ts#L70
That function as it currently exists would need a lot of work to merely be wrong.
OK, here is an imperfect workaround. npm i @cto.af/textdecoder
. Then at the top, change to:
import TextDecoder from "@cto.af/textdecoder/polyfill";
global.TextDecoder = TextDecoder;
import {fixes} from "cbor-rn-prereqs";
import * as cbor from "cbor";
import * as utils from "cbor/lib/utils.js"
const td = new TextDecoder('utf8', {fatal: true, ignoreBOM: true})
utils.utf8 = buf => td.decode(buf)
Unfortunately, the top level TextDecoder instance in cbor
is resolved at compile time, so replacing the utf8
function is the quickest way to get you up and running. I did a release of @cto.af/textdecoder
today that had a full and tested UTF-8 implementation, so that's what you're swapping in.
I would also recommend filing a bug against NativeScript to have them fix their quite broken implementation. It doesn't decode non-ASCII UTF-8, and doesn't deal with slices.
@hildjj Thanks for your Info. Before I opened this issue, I tried setting the variable as follow global.TextDecoder = util.TextDecoder
but TextDecoder is read only. Did I miss something? I will try your approach monday.
util.TextDecoder only exists in node 10, I think. Setting global.TextDecoder
isn't really needed for cbor
, but I left it there in case you're using it somewhere else.
I'll see what I can do to fix this in cbor
, but short of running a set of unit tests at runtime, I'm not sure how to work around an existing-but-broken TextDecoder implementation. I might be able to make it easier to replace, at least.
Did we get this solved adequately?
I think we came to an adequate solution. File more bugs if you continue to have problems with NativeScript being broken.
I want to use this library in nativescript/angular. I got it bundled and working as follow:
The same behavior when using
cbor-web
I would appreciate any help.