Closed yurivict closed 5 days ago
Hmm, do you have all the dependencies installed?
Have you done a pip install -r requirements-dev.txt
before running the tests @yurivict ?
@seperman
Currently many tests fail because of quotation mark differences, like this:
def prep_str(obj, ignore_string_type_changes=True):
- return obj if ignore_string_type_changes else 'str:{}'.format(obj)
+ return obj if ignore_string_type_changes else "str:{}".format(obj)
What might be causing this?
There are no specific import
statement failures, so the problem isn't that some dependencies aren't present.
Hi,That is weird. Is that a bug in your FreeBSD installed version of Python? Going from single quotes to double quotes should not matter.Sep DehpourOn Oct 20, 2023, at 4:32 PM, @.*** wrote: @seperman Currently many tests fail because of quotation mark differences, like this: def prep_str(obj, ignore_string_type_changes=True):
What might be causing this? There are no specific import statement failures, so the problem isn't that some dependencies aren't present.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>
Is that a bug in your FreeBSD installed version of Python?
I am not aware of any failures with that type of symptoms.
We have about 5,500+ Python ports in FreeBSD, and I've never seen anything like this.
Version 7.0.1 has 3 tests failing:
==================================================================================== test session starts ====================================================================================
platform freebsd14 -- Python 3.9.18, pytest-7.4.4, pluggy-1.4.0
Using --randomly-seed=1314162047
rootdir: /usr/ports/devel/py-deepdiff/work-py39/deepdiff-7.0.1
configfile: pytest.ini
plugins: anyio-4.3.0, hypothesis-6.98.18, datadir-1.5.0, cov-4.1.0, flake8-1.1.1, randomly-3.12.0, timeout-2.1.0, time-machine-2.11.0, rerunfailures-11.1.2, flaky-3.7.0, forked-1.6.0, aspectlib-2.0.0, xdist-3.5.0, env-0.6.2, mock-3.10.0, subtests-0.12.1, parallel-0.1.1
collected 915 items
tests/test_anyset.py ..... [ 0%]
tests/test_diff_tree.py ..............s [ 2%]
tests/test_delta.py ..F............................................................................................................. [ 14%]
tests/test_diff_math.py .... [ 14%]
tests/test_path.py .................. [ 16%]
tests/test_command.py ....................... [ 19%]
tests/test_hash.py ................................................................................................... [ 30%]
tests/test_cache.py .ss... [ 30%]
tests/test_model.py ...................... [ 33%]
tests/test_helper.py ................................................................................................................................................................ [ 50%]
.. [ 50%]
tests/test_diff_text.py .............s..........s.......................................................................................................................s............ [ 68%]
........ [ 68%]
tests/test_operators.py ........ [ 69%]
tests/test_diff_datetime.py ... [ 70%]
tests/test_search.py ............................................................. [ 76%]
tests/test_serialization.py .......................F........F............................ [ 83%]
tests/test_distance.py ...................................... [ 87%]
tests/test_diff_other.py ............. [ 89%]
tests/test_lfucache.py .... [ 89%]
tests/test_diff_numpy.py .............. [ 91%]
tests/test_ignore_order.py ..............s................ss........................................s........ [100%]
========================================================================================= FAILURES ==========================================================================================
_________________________________________________________________ TestDeltaCompareFunc.test_list_of_alphabet_and_its_delta __________________________________________________________________
self = <tests.test_delta.TestDeltaCompareFunc object at 0x1f2399a0d5b0>
def test_list_of_alphabet_and_its_delta(self):
l1 = "A B C D E F G D H".split()
l2 = "B C X D H Y Z".split()
diff = DeepDiff(l1, l2)
# Problem: The index of values_changed should be either all for AFTER removals or BEFORE removals.
# What we have here is that F & G transformation to Y and Z is not compatible with A and E removal
# it is really meant for the removals to happen first, and then have indexes in L2 for values changing
# rather than indexes in L1. Here what we need to have is:
# A B C D E F G D H
# A B C-X-E
# B C D F G D H # removal
# What we really need is to report is as it is in difflib for delta specifically:
# A B C D E F G D H
# B C D E F G D H delete t1[0:1] --> t2[0:0] ['A'] --> []
# B C D E F G D H equal t1[1:3] --> t2[0:2] ['B', 'C'] --> ['B', 'C']
# B C X D H replace t1[3:7] --> t2[2:3] ['D', 'E', 'F', 'G'] --> ['X']
# B C X D H equal t1[7:9] --> t2[3:5] ['D', 'H'] --> ['D', 'H']
# B C X D H Y Z insert t1[9:9] --> t2[5:7] [] --> ['Y', 'Z']
# So in this case, it needs to also include information about what stays equal in the delta
# NOTE: the problem is that these operations need to be performed in a specific order.
# DeepDiff removes that order and just buckets all insertions vs. replace vs. delete in their own buckets.
# For times that we use Difflib, we may want to keep the information for the array_change key
# just for the sake of delta, but not for reporting in deepdiff itself.
# that way we can re-apply the changes as they were reported in delta.
delta = Delta(diff)
assert l2 == l1 + delta
with pytest.raises(ValueError) as exc_info:
l1 == l2 - delta
assert "Please recreate the delta with bidirectional=True" == str(exc_info.value)
delta2 = Delta(diff, bidirectional=True)
assert l2 == l1 + delta2
assert l1 == l2 - delta2
dump = Delta(diff, bidirectional=True).dumps()
delta3 = Delta(dump, bidirectional=True)
assert l2 == l1 + delta3
assert l1 == l2 - delta3
dump4 = Delta(diff, bidirectional=True, serializer=json_dumps).dumps()
> delta4 = Delta(dump4, bidirectional=True, deserializer=json_loads)
tests/test_delta.py:2395:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
deepdiff/delta.py:130: in __init__
self.diff = _deserializer(diff, safe_to_import=safe_to_import)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = '{"values_changed": {"root[3]": {"new_value": "X", "old_value": "D", "new_path": "root[2]"}, "root[6]": {"new_value": ...3, 7, 2, 3, ["D", "E", "F", "G"], ["X"]], ["equal", 7, 9, 3, 5, null, null], ["insert", 9, 9, 5, 7, [], ["Y", "Z"]]]}}'
safe_to_import = None
def _deserializer(obj, safe_to_import=None):
result = deserializer(obj)
if result.get('_iterable_opcodes'):
_iterable_opcodes = {}
for path, op_codes in result['_iterable_opcodes'].items():
_iterable_opcodes[path] = []
for op_code in op_codes:
_iterable_opcodes[path].append(
> Opcode(
**op_code
)
)
E TypeError: deepdiff.helper.Opcode() argument after ** must be a mapping, not list
deepdiff/delta.py:102: TypeError
_____________________________________________________________________ TestDeepDiffPretty.test_namedtuple_seriazliation ______________________________________________________________________
self = <tests.test_serialization.TestDeepDiffPretty object at 0x1f2399d4a670>
def test_namedtuple_seriazliation(self):
op_code = Opcode(tag="replace", t1_from_index=0, t1_to_index=1, t2_from_index=10, t2_to_index=20)
serialized = json_dumps(op_code)
expected = '{"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}'
> assert serialized == expected
E assert '["replace", ..., null, null]' == '{"tag":"repl...values":null}'
E - {"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}
E + ["replace", 0, 1, 10, 20, null, null]
tests/test_serialization.py:362: AssertionError
______________________________________________________________ TestDeepDiffPretty.test_json_dumps_and_loads[8-value7-<lambda>] ______________________________________________________________
self = <tests.test_serialization.TestDeepDiffPretty object at 0x1f2399d4a4c0>, test_num = 8
value = SomeStats(counter=Counter({'a': 2, 'b': 1}), context_aware_counter=None, min_int=0, max_int=10), func_to_convert_back = <function TestDeepDiffPretty.<lambda> at 0x1f2399d0ca60>
@pytest.mark.parametrize('test_num, value, func_to_convert_back', [
(1, {'10': None}, None),
(2, {"type_changes": {"root": {"old_type": None, "new_type": list, "new_value": ["你好", 2, 3, 5]}}}, None),
(3, {'10': Decimal(2017)}, None),
(4, Decimal(2017.1), None),
(5, {1, 2, 10}, set),
(6, datetime.datetime(2023, 10, 11), datetime.datetime.fromisoformat),
(7, datetime.datetime.utcnow(), datetime.datetime.fromisoformat),
(8, field_stats1, lambda x: SomeStats(**x)),
(9, np.array([[ 101, 3533, 1998, 4532, 2024, 3415, 1012, 102]]), np.array)
])
def test_json_dumps_and_loads(self, test_num, value, func_to_convert_back):
if test_num == 8 and py_current_version < 3.8:
print(f"Skipping test_json_dumps_and_loads #{test_num} on Python {py_current_version}")
return
serialized = json_dumps(value)
back = json_loads(serialized)
if func_to_convert_back:
> back = func_to_convert_back(back)
tests/test_serialization.py:352:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
x = [{'a': 2, 'b': 1}, None, 0, 10]
> (8, field_stats1, lambda x: SomeStats(**x)),
(9, np.array([[ 101, 3533, 1998, 4532, 2024, 3415, 1012, 102]]), np.array)
])
E TypeError: tests.test_serialization.SomeStats() argument after ** must be a mapping, not list
tests/test_serialization.py:342: TypeError
================================================================================== short test summary info ==================================================================================
FAILED tests/test_delta.py::TestDeltaCompareFunc::test_list_of_alphabet_and_its_delta - TypeError: deepdiff.helper.Opcode() argument after ** must be a mapping, not list
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_namedtuple_seriazliation - assert '["replace", ..., null, null]' == '{"tag":"repl...values":null}'
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_json_dumps_and_loads[8-value7-<lambda>] - TypeError: tests.test_serialization.SomeStats() argument after ** must be a mapping, not list
======================================================================== 3 failed, 902 passed, 10 skipped in 10.61s =========================================================================
*** Error code 1
@yurivict One of the reason we stopped supporting Python 3.7 was that NamedTuple serialization in 3.7 was returning a list instead of a dictionary. That was for Python 3.7 installed on Debian via pyenv.
The error you see that says TypeError: deepdiff.helper.Opcode() argument after ** must be a mapping, not list
makes me think you are using an old version of Python on FreeBSD which is returning a list for NamedTuples when serialized. Is that true?
We are using Python-3.9.
There are other failures in 8.0.1
@yurivict I just left a comment on the other ticket you opened on Orderly-set. I have never used FreeBSD before and I find it unexpected that Python behaves differently on FreeBSD vs. Linux. Did the issue fix on its own?
Version: 5.8.1 Python-3.8 FreeBSD 13.1