aliyun / alibabacloud-odps-maxframe-client

Apache License 2.0
3 stars 1 forks source link

astype error: Data invalid: ODPS-0010000:InvalidArgument #13

Open snowljs opened 2 weeks ago

snowljs commented 2 weeks ago
Traceback (most recent call last):
  File "<pyodps_user_code>", line 30, in <module>
    res= ser.execute().fetch()
  File "/home/tops/lib/python3.7/site-packages/maxframe/core/entity/tileables.py", line 434, in execute
    result = self.data.execute(session=session, **kw)
  File "/home/tops/lib/python3.7/site-packages/maxframe/core/entity/executable.py", line 150, in execute
    return execute(self, session=session, **kw)
  File "/home/tops/lib/python3.7/site-packages/maxframe/session.py", line 1061, in execute
    **kwargs,
  File "/home/tops/lib/python3.7/site-packages/maxframe/session.py", line 873, in execute
    timeout=self._isolated_session.timeout
  File "/home/tops/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/home/tops/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/home/tops/lib/python3.7/site-packages/maxframe/utils.py", line 976, in cb
    dest.set_result(fut.result())
  File "/home/tops/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/home/tops/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/home/tops/lib/python3.7/site-packages/maxframe/session.py", line 1007, in _execute
    execution_info = await session.execute(*tileables, **kwargs)
  File "/home/tops/lib/python3.7/site-packages/maxframe_client/session/odps.py", line 229, in execute
    source_replacements = self._scan_and_replace_pandas_sources(tileable_graph)
  File "/home/tops/lib/python3.7/site-packages/maxframe/core/mode.py", line 77, in _inner
    return func(*args, **kwargs)
  File "/home/tops/lib/python3.7/site-packages/maxframe_client/session/odps.py", line 187, in _scan_and_replace_pandas_sources
    replaced = self._upload_and_get_read_tileable(t)
  File "/home/tops/lib/python3.7/site-packages/maxframe_client/session/odps.py", line 159, in _upload_and_get_read_tileable
    writer.write(arrow_batch)
  File "/home/tops/lib/python3.7/contextlib.py", line 119, in __exit__
    next(self.gen)
  File "/home/tops/lib/python3.7/site-packages/maxframe/odpsio/tableio.py", line 294, in open_writer
    commit_msg = writer.close()
  File "/home/tops/lib/python3.7/site-packages/maxframe/odpsio/tableio.py", line 166, in close
    commit_msg, is_success = self._writer.finish()
  File "/home/tops/lib/python3.7/site-packages/odps/apis/storage_api/storage_api.py", line 558, in finish
    return self._sink.finish()
  File "/home/tops/lib/python3.7/site-packages/odps/apis/storage_api/storage_api.py", line 425, in finish
    self._res = self._req_io.finish()
  File "/home/tops/lib/python3.7/site-packages/odps/tunnel/io/stream.py", line 129, in finish
    self._reraise_errors()
  File "/home/tops/lib/python3.7/site-packages/odps/tunnel/io/stream.py", line 84, in _reraise_errors
    six.reraise(ex_type, ex_value, tb)
  File "/home/tops/lib/python3.7/site-packages/odps/lib/six.py", line 719, in reraise
    raise value
  File "/home/tops/lib/python3.7/site-packages/odps/tunnel/io/stream.py", line 76, in _async_func
    self._resp = self._post_call(self.data_generator())
  File "/home/tops/lib/python3.7/site-packages/odps/apis/storage_api/storage_api.py", line 792, in upload
    return self.tunnel_rest.post(url, data=data, params=params, headers=headers)
  File "/home/tops/lib/python3.7/site-packages/odps/rest.py", line 290, in post
    return self.request(url, 'post', data=data, **kwargs)
  File "/home/tops/lib/python3.7/site-packages/odps/rest.py", line 198, in request
    return self._request(url, method, stream=stream, **kwargs)
  File "/home/tops/lib/python3.7/site-packages/odps/rest.py", line 282, in _request
    errors.throw_if_parsable(res, self._endpoint, self._tag)
  File "/home/tops/lib/python3.7/site-packages/odps/errors.py", line 100, in throw_if_parsable
    raise parse_response(resp, endpoint, tag)
odps.errors.ODPSError: ODPS-0422224: RequestId: 202406191657538be1db0b00000654 Tag: TUNNEL Endpoint: http://dt.odps.aliyun.com
ODPS-0422224: Write data failed - The data stream you provided was not well-formed or did not validate against schema. WriteRecordBatch failed. Data invalid: ODPS-0010000:InvalidArgument:int32 is not equal to int64

maxframe code

import maxframe.dataframe as md                                    
df = md.DataFrame(pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]}))
print(df.dtypes)

print(df.astype({'col1': 'int32'}).dtypes)