wenleix / StructTorch

Common and composable data structures built on PyTorch Tensor for efficient batch data representation and processing in PyTorch model authoring.
BSD 3-Clause "New" or "Revised" License
5 stars 1 forks source link

Proxy update #20

Closed dracifer closed 1 year ago

dracifer commented 1 year ago

update proxy and trace. still very dirty and rudimental. create as a checkpoint and will clean up later

dracifer commented 1 year ago

not intend to add to the original repo before complet~~

wenleix commented 1 year ago

output graph

===Graph===
  index  name                 op             target                                                        args                                                    kwargs
-------  -------------------  -------------  ------------------------------------------------------------  ------------------------------------------------------  -------------------------------------------------------
      0  arg0_0               placeholder    arg0                                                          ()                                                      {}
      1  arg1_0               placeholder    arg1                                                          ()                                                      {}
      2  struct_column_0      call_function  <class 'axolotls.struct_column.StructColumn'>                 ()                                                      {'field_columns': {'f1': arg0_0, 'f2': arg1_0}}
      3  struct_column_1      call_function  <class 'axolotls.struct_column.StructColumn'>                 ()                                                      {'field_columns': {'float_features': struct_column_0}}
      4  getitem_0            call_function  <built-in function getitem>                                   (struct_column_1, 'float_features')                     {}
      5  getitem_1            call_function  <built-in function getitem>                                   (getitem_0, 'f1')                                       {}
      6  udf_normalize_0      call_function  <function udf_normalize at 0x7f564a744310>                    (getitem_1, 1.0, 0.1)                                   {}
      7  getitem_2            call_function  <built-in function getitem>                                   (struct_column_1, 'float_features')                     {}
      8  setitem_0            call_function  <built-in function setitem>                                   (getitem_2, 'f1', udf_normalize_0)                      {}
      9  getitem_3            call_function  <built-in function getitem>                                   (struct_column_1, 'float_features')                     {}
     10  getitem_4            call_function  <built-in function getitem>                                   (getitem_3, 'f2')                                       {}
     11  getattr_0            call_function  <built-in function getattr>                                   (getitem_4, 'values')                                   {}
     12  _tensor_constant0_0  get_attr       _tensor_constant0                                             tensor([4.0000, 5.0000, 7.1000, 9.0000])                {}
     13  bucketize_0          call_function  <built-in method bucketize of type object at 0x7f5649738740>  (getattr_0, _tensor_constant0_0)                        {}
     14  numeric_column_0     call_function  <class 'axolotls.numeric_column.NumericColumn'>               ()                                                      {'values': bucketize_0}
     15  numel_0              call_method    numel                                                         (bucketize_0,)                                          {}
     16  add_0                call_function  <built-in function add>                                       (numel_0, 1)                                            {}
     17  torch_arange_0       call_function  <function torch_arange at 0x7f55a37f3dc0>                     (0, add_0)                                              {}
     18  list_column_0        call_function  <class 'axolotls.list_column.ListColumn'>                     ()                                                      {'values': numeric_column_0, 'offsets': torch_arange_0}
     19  struct_column_2      call_function  <class 'axolotls.struct_column.StructColumn'>                 ()                                                      {'field_columns': {'derived_id2': list_column_0}}
     20  setitem_1            call_function  <built-in function setitem>                                   (struct_column_1, 'id_list_features', struct_column_2)  {}
     21  output_0             output         output                                                        ([struct_column_1],)                                    {}
wenleix commented 1 year ago

PR in the forked repo: https://github.com/dracifer/axolotls/pull/1