Temporarily return to rapidjson

[josejachuf]

I have a problem that is not directly from emmett but with the installation of orjson.

All the server infrastructure I use is freebsd (+ jails). The orjson installation with pip does not use binaries, it is compiled. To compile you need to install some dependencies, including maturin. In the maturin installation an error occurs (I reported it) and therefore I cannot install orjson.

I've been seeing this [1]

What I want to do is temporarily modify serializers.py [2] and go back to rapidjson. This is apparently the only place where orjson is referenced. The query is if this change is enough and nothing will break.

[1] https://github.com/emmett-framework/emmett/commit/2878fbdfd0ddedbee40ed863d78fb9c6525fc93e [2] https://github.com/emmett-framework/emmett/commit/5bbbae35c8517fd147e9b76ec4486432c347ef9e

[josejachuf]

I modified: [1] imports and json = partial(

and [2] class JSONServicePipe(Pipe):

output = 'str' (with orjosn output = 'bytes')

and in a simple example it works

[1] emmett/serializers.py [2] emmett/tools/service.py

[gi0baro]

@josejachuf yes it should be enough.

I'm thinking about a patch-release to make orjson an extra, will update you if I decide in that direction

[josejachuf]

Hi @gi0baro. This how it would be, a parameter would be passed to App () when instantiating the application to tell it to use orjson?

[gi0baro]

@josejachuf I was thinking more to extra-packages, eg: pip install emmett[orjson]

[josejachuf]

Ok. And then use try...except imports?

[gi0baro]

@josejachuf yep, I can't think about a better approach.

btw, I still need to think about this, AFAIK on the main platforms the only issue right now with orjson is the lack of wheels for windows and macos on Python 3.9 (that should be addressed within next 2 weeks I hope).

[josejachuf]

Hi @gi0baro, Here in service.py:

class JSONServicePipe(Pipe):
    __slots__ = ['decoder', 'encoder']
    # output = 'bytes'
    output = 'str'

is the library used (orjson / rapidjson) that defines the data type?

[gi0baro]

@josejachuf yes, orjson returns byte strings, while rapidjson returns unicode strings

[gi0baro]

@josejachuf with v2.1.1 orjson is optional. You can upgrade and just have the same behaviour of 2.0.x (ensure to cleanup your virtualenv eventually).

I'm closing this.

[josejachuf]

Thanks @gi0baro

[josejachuf]

I received help to install maturin in freebsd and already with maturin installed orjson it installed without problems. in this link [1] is the solution in case someone needs it. Basically in a new installation you should do the following

pkg install -y rust-nightly gcc9 python38

Here's the trick:

cd /usr/bin
mv ld ld.bak
ln -s /usr/local/bin/ld /usr/bin/ld

Then in your virtualenv simply:

pip install maturin

[1] https://github.com/PyO3/maturin/issues/363

[rozhuk-im]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266213