Open albertvillanova opened 2 years ago
The superset dataset (Chinese Gigaword Fifth Edition) that contains this dataset also seems to be available from Uni Tübingen. (Access is restricted however, but it might help to reach out)
Chinese Gigaword Fifth Edition is from 2011, still better than nothing though.
Tübingen has been a member of the LDC in 2011. As a member institution, we were able to receive a free copy of this data set. The standard license does not permit member institutions to share the data with researchers who are not part of the respective organization. One solution to such issues may be to turn to derived data formats that do not touch upon copyright and privacy, but this is an ongoing discussion, also within the German national research data infrastructure. These derived formats would probably not be suitable for training in machine learning. Another option for legally using the data could be to take the algorithms to the data, i.e. run for example training software at the data storage location under their license. However, this would require to carefully monitor the code for security reasons and evaluate the output to make sure that the licenses are not violated.
@ttrippel Thank you for taking the time to answer!
I think we can keep this dataset out of the target datasets for the moment...