CPSSD / LUCAS

The repository for the LUCAS/Lucify project
MIT License
11 stars 4 forks source link

Add generating TF-Record files with reviewer data #184

Closed StefanKennedy closed 5 years ago

StefanKennedy commented 5 years ago

This notebook generates TF-Record files containing the features used in a convolutional network. The reason we need this is because it allows us to performant-ly feed data to the model without having it all loaded in memory at once.

It also includes reviewer features, which we know increases the accuracy of the Convolutional Network when concatenated.

Changing the max_review_words value changes the limit on how long a review we will include. All reviews must be padded to this max length so this has memory implications. Output files contain this value in their name to identify them.

StefanKennedy commented 5 years ago

I'm keeping it separate to use it as the test set