amazon-ion / ion-hash-python

A Python implementation of Amazon Ion Hash.
https://amazon-ion.github.io/ion-hash/
Apache License 2.0
10 stars 7 forks source link

Amazon Ion Version Pinned to 0.4.0 #10

Closed almann closed 5 years ago

almann commented 5 years ago

setup.py is pinned to amazon.ion==0.4.0 which is too strict and doesn't work with the current version of Ion python.

almann commented 5 years ago

Attempted the fix in commit d6742fa, but the tests break

I think amzn/ion-python#97 appears to break the tests (since it is the only change in 0.4.1 that remotely seems similar), but I am still unclear as to the problem. Specifically around our usage of the event:

FAILED
tests/test_ion_hash_tests.py:125 (test_binary[132ion_test132])
ion_test = OrderedDict([('ion', <amazon.ion.core.MultimapValue object at 0x10cece3d0>), ('expect', <amazon.ion.core.MultimapValue object at 0x10ced2310>)])

    @pytest.mark.parametrize("ion_test", _test_data("identity"), ids=_test_name)
    def test_binary(ion_test):
        _run_test(ion_test,
                  _consumer_provider(_reader_provider("binary"),
>                                    _to_buffer(ion_test, binary=True)))

tests/test_ion_hash_tests.py:130: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_ion_hash_tests.py:67: in _to_buffer
    v = ion.dumps(ion_test['ion'], binary=binary)
venv/lib/python3.7/site-packages/amazon/ion/simpleion.py:260: in dumps
    ignore_nan=ignore_nan, int_as_string_bitcount=int_as_string_bitcount, iterable_as_array=iterable_as_array)
venv/lib/python3.7/site-packages/amazon/ion/simpleion.py:146: in dump
    _dump(obj, writer)
venv/lib/python3.7/site-packages/amazon/ion/simpleion.py:198: in _dump
    _dump(val, writer, field)
venv/lib/python3.7/site-packages/amazon/ion/simpleion.py:209: in _dump
    writer.send(event)
venv/lib/python3.7/site-packages/amazon/ion/writer.py:150: in blocking_writer
    for result_event in _drain(writer, ion_event):
venv/lib/python3.7/site-packages/amazon/ion/writer.py:129: in _drain
    result_event = writer.send(ion_event)
venv/lib/python3.7/site-packages/amazon/ion/writer.py:111: in writer_trampoline
    trans = trans.delegate.send(Transition(ion_event, trans.delegate))
venv/lib/python3.7/site-packages/amazon/ion/writer_binary.py:214: in _managed_binary_writer_coroutine
    write_event = value_writer.send(ion_event)
venv/lib/python3.7/site-packages/amazon/ion/writer.py:111: in writer_trampoline
    trans = trans.delegate.send(Transition(ion_event, trans.delegate))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

writer_buffer = <amazon.ion.writer_buffer.BufferTree object at 0x10dc1ec90>
depth = 1
container_event = IonEvent(event_type=<IonEventType.CONTAINER_START: 2>, ion_type=<IonType.STRUCT: 12>, value=OrderedDict([(None, <amazon.ion.core.MultimapValue object at 0x10cecea50>)]), field_name=None, annotations=(), depth=None)
whence = <generator object _raw_writer_coroutine at 0x10f57b250>
pending_annotations = ()

    @coroutine
    def _raw_writer_coroutine(writer_buffer, depth=0, container_event=None,
                              whence=None, pending_annotations=None):

        def fail():
            raise TypeError('Invalid event: %s at depth %d' % (ion_event, depth))

        write_result = None
        while True:
            ion_event, self = (yield write_result)
            delegate = self
            curr_annotations = ion_event.annotations
            writer_event = _WRITER_EVENT_NEEDS_INPUT_EMPTY
            if depth > 0 and container_event.ion_type is IonType.STRUCT \
                    and ion_event.event_type.begins_value:
                # A field name symbol ID is required at this position.
                sid_buffer = bytearray()
>               _write_varuint(sid_buffer, ion_event.field_name.sid)  # Write the field name's symbol ID.
E               AttributeError: 'NoneType' object has no attribute 'sid'
pbcornell commented 5 years ago

The breakage shown above is the result of a change in behavior in ion-python's IonEvent.field_name, and has been addressed by https://github.com/amzn/ion-python/pull/105.

There is another, small, breaking change to an internal ion-python API that will be addressed shortly, along with bumping ion-hash-python's dependency to ion-python >= 0.5.