`rename_table`'s `to` fails when using a variable that references another variable with a dot

[josegalarza]

rename_table>:'s to does not work if uses a variable that references another variable which has a . (dot) in it.

For example, if I have a code like the following:

+demo:
  for_each>:
    params: [{from_name: 'foo', to_name: 'bar'}]
  _do:
    _export:
      from_name: ${params.from_name}
      to_name: ${params.to_name}
    td_ddl>:
    rename_tables: [{from: ${from_name}, to: ${to_name}}] # this will fail

I get the error: 'to' option of rename_tables can't include database name (config).

I think the error comes from this line: https://github.com/treasure-data/digdag/blob/19dfc0c00e980a6d2c7dfb0f65cc66d62ac63268/digdag-standards/src/main/java/io/digdag/standards/operator/td/TdDdlOperatorFactory.java#L115

PS: I know that if instead of: rename_tables: [{from: ${from_name}, to: ${to_name}}], rename_tables: [{from: ${from_name}, to: ${params.to_name}}] it will work, but still. This issue occurred to me when I had a second workflow which basically does the td_ddl>: part of the example I used.

[hiroyuki-sato]

Hello, @josegalarza

This issue may better to contact to Arm Treasure Data. (support@treasure-data.com) They are deploying Digdag as Treasure Workflow. And td_ddl is TD specific operator.

[muga]

@josegalarza @hiroyuki-sato The error message that you got means that the operator doesn't support renaming tables across databasees. As you might know already, the following definition could work for you.

_export:
  td:
    database: my_database

+ddl_demo:
  for_each>:
    params: [{from_name: 'foo01', to_name: 'bar01'}, {from_name: 'foo02', to_name: 'bar02'}]
  _do:
    td_ddl>:
      rename_tables: [{from: "${params.from_name}", to: "${params.to_name}"}]

This issue occurred to me when I had a second workflow which basically does the td_ddl>: part of the example I used.

I'm still confusing the above your comment a little bit. Please tell us more concrete example if you can.