Closed affanmehmood closed 2 years ago
lvyiwei1
commented
2 years ago Thank you for your interest in our work!
When we trained our model, we add a
Also, we didn't try to train complex neural networks (like GPT) on lexical transfers since they can most efficiently (and accurately) done by simple rule-based word replacements, so maybe you can try to use rule-based programs to perform lexical transfers instead.
Hope that helps!
affanmehmood
commented
2 years ago Actually, I'm using the script "Model Codes/GPT-Baseline.py" that is present in this repo to train, and I can see that there are "endoftext" being appended.
lvyiwei1
commented
2 years ago In that case, maybe try a less complex neural network, an editor model (like retrieve-edit) or just use a rule-based program instead?
I trained the GPT-Baseline on 10,000 epochs and a batch size of 20 with the MFS dataset that I created using single_transform_checkout.py. I'm getting weird output. Is there a way to improve the training?
Input 1: The weather in 1816 Europe was abnormally wet, keeping many inhabitants indoors that summer. From April until September of that year, "it rained in Switzerland on 130 out of the 183 days from April to September" (Phillips, 2006). Unlike today, one could not simply turn on a television or click through the Internet in order to entertain oneself. Instead, it was much more common for the educated people of the day to spend time reading, discussing well-known authors and artists of the day, playing at cards and walking in their gardens and walking paths.
Output 1: The weather in 1816 Europe was abnormally wet, keeping many inhabitants indoors that summer. From April until September of that year, "it rained in Switzerland on 130 out of the 183 days from April to September" (Phillips, 2006). Unlike today, one could not simply turn on a television or click through the Internet in order to entertain oneself. Instead, it was much more common for the educated people of the day to spend time reading, discussing well-known authors and artists of the day, playing at cards and walking in their gardens and walking paths. Sum Set Column Never Tort Code Location Spread Order Column Speed Trap Box Points Lists Location Tort Method Mouse Number Maps Never Code Location Copy File Track Track Drill Scores File Place Location Character Character Walk Drive List Player Character Base Direction Dog Location Mouse Scores Quality Base Character Base Location File Maps Clock Speed Trap Way Too Tracks Track Notes Way Action Dog Gas Enemy Place List Performance Map Player Drill Action Speed D Direction Enemy Walk Order Number Enemy Enemy Cause Base Enemy Drill Never D Maps Base Performance File Things Character Players Players Trap Player Other Kill Track Walk Drive Enemy Actions Speed Action Score Tracks Action Mouse Description Base Player Local Place Other File Network Place Other Position: Other Drill Order Order Order: Action Drill Drive File Place Customer Level Frequency Index Base Set Speed Trap Local Drill Place Stand Trap Number Level Cause Maps P Local Stand Dog, Customer Tracks Report Map Other Performance Character In Tracks Local Copy Spot Update File The Trap Order Mouse Spot Track Stand Track In Report Tracks Position A No Name Performance In Trap Trap Players D Perform The Trap Score D Mouse Trad Trap Tracks Foot Name Door File Method Order Frequency Door Door File Player Jump Track Track Trad Other Column Feet Score Peg Feet File Position Position Stand Trap Performance Feet Raise Rule Copy Method Drive Structure Trap Trap Tactics Drive Feet Trap Trap Trap Drill Drill File Track Trap Drill Gems Trap Trap Trap Drill Structure Pack Teams Drive Enemy Trap Trap Peg Trap Trap Trap Drive Spot Monster Tracker Trap Trap Trap Door Squ Trap Trap Structure Number Drill Drive Guys Drill Drill Move Trap Duel File Trap Drill Drill Tactics Trap Trap Trap Track Drill Drill Drill Drill Drill Track Trap Trap Drive Trap Feet Enemy Drill Drill Tracker Trap Trap Drill Drill Trap Sheet Drill Drill Drill Drill Trap Trap Drill Drill Drill Tape Trap Drill Drill Drill Drill Spot Base Drill Drill Drill Duel Spot Sheet Trap Trap Game Trap Double Trap Order Exhibit Sheet Stand Trap Monster Trap Trap Trap Trap Trap Trap File Trap Duel Speed Trap Trap Trap Spot Gems Fine Drill Spot Trap Slow Trap Trap Drill Drill Drill Tracks Sheet Trap Enemy Drill Trap File Speed Trap Player Tracker Tracker Table Speed Track Drive Trap Trap File Speed Speed Method Copy Structure Analy Track Enemy Drive Structure Character Speed Order Order Enemy Guide Drill Drill Trap Trap Trap Trap Drive Character Trap Trap Tracks Trap Drive File Actions Look Number Trap Number Drill Monster Treasure Treasure Copy Enemy Base Foot Foot Position Character Drill Way Location Folder Speed Drill Track Package Table Console Player Fine Cause Tail Copy Way Cause Structure Engine Trap Trap Team Trap Copy Drill Ground Analy Mouse Guide Foot Tactics Drill Tail Mouse Enemy Drive Order Track Guide Column Squ Drive Report Ground Foot Look Location Field Number Table Order Command Work File Copy Speed Spread Guide Console Character File Copy Spread Size Slow Fine Drive Trap Speed File File Fine Base Way Base Move Each Copy Order Cause Look Location Actions Base Base Low Order Position Action Set Move Speed Each Cause Player List Tem Table Player Set Drive Base Mail Drill Trap Base Tem Head Speed Drive Drive File Column Drive Action Speed Head Source Track Foot Trap Copy Track Spread Tactics Fine Foot Mail Drive Mouse Each Foot Speed Position Report Report Foot Location Map Ground Trap Copy Track Trap Trap Character Cause Cause Move Nothing Fine Number Drill File Drill Drill Trap Level Cause Base Way Slow Player Fine Nothing Size Character Position Speed Squ Report Enemy Move Map Map Speed Position Salary Fine File Drive Track File Cause Location Position Order Way Stand Character Cold Stand Team Players Action Sign Source Slow Squ List Fast Position Player Squ Past Work Players Location Trap Drill Source Slow Less Salary Trap Team Stand Training File Speed Family Smart Location Done Salary Track Done Foot Slow Drive Character Speed Work Salary Enemy Map Stand Actions Things Trap Trend Position Position Score Trap Ground Trap Player Ground Drill Fast Tem Player Order Character Level Ground List Base Position Squ Smart Head Door Track Head Trap Trap Players Tem Trap Less Spread Base Plot Score Player Stand Report Tail Actions Position Plot File Action Trap Drive Trap Spread Plot Way Way Track Nothing Others Less Location Each Head Character Slow Tail Event Structure Structure Smart Head Trap Event Copy Actions Order Level Player Fast Trap Speed Drive Drive Through Stand Speed Family Door Base Track Stand Move Tail Copy Location Player Door Dim Base Ground Stand Fine Location Team Position Past Enemy Position Foot Past Drive Head Score Copy Score Plot Dim Copy Fine Speed Head Plot Move Tail Way Actions Head Structure Slow Head Through Track Nothing Physical Position Smart Character Head Way Drive Work Structure None Level Player Fine Base Table Fat Event Score Poor Each Actions Console Fast Stand Team Team Team Low Physical Character Smart Each Copy Slow Together Expl Past Size Copy Base Stand Position Track Team Head Ground Nothing Each Base Level Position Fast Character Drive Physical Location Together Nothing Size Head Base Location Spread Location Fat Way Fine Low Spread Level Console None Spread File Player Move Fine Size Tem Low Fine Source Through Base Copy Location Same Spread Stat Character Level Score Fast Fat Stand Score Fine Position Player Stand
Input 2: If you were Mary Shelley in the company of Byron and others, you amused each other by reading out loud, sharing a common interest in a particular book, and sharing with the others your own writing. In her introduction to Frankenstein, her explanation of how this extraordinary novel came to be was due, at least in part, to the weather and the company (Shelley, 1816). "I passed the summer of 1816 in the environs of Geneva. The season was cold and rainy, and ...we occasionally amused ourselves with some German stories of ghosts... These tales excited in us a playful desire of imitation" (Shelley, as quoted in Phillips, 2006).
Output 2: If you were Mary Shelley in the company of Byron and others, you amused each other by reading out loud, sharing a common interest in a particular book, and sharing with the others your own writing. In her introduction to Frankenstein, her explanation of how this extraordinary novel came to be was due, at least in part, to the weather and the company (Shelley, 1816). "I passed the summer of 1816 in the environs of Geneva. The season was cold and rainy, and...we occasionally amused ourselves with some German stories of ghosts... These tales excited in us a playful desire of imitation" (Shelley, as quoted in Phillips, 2006). List Thing Enemy Trap Dog Dro Speed,, Performance Method Never Track Mouse Command Dro Number Never Trap Skill Number Too Fun Method Map Guide D Maps Action Place Character Enemy D Enemy Enemy Spread Set Tem Action Dog Drive Performance Speed Done Too Trap Speed Enemy Too Mouse Things Cause Dog File Dead Index Order Att Foot Trap Spread Tactics Move Character Player Action Dog Character Kill List Head Trap Order Trap Way Players Too Dro Wind Number Place D D Place Less Character Dog Things Drive Order Stat Foot Things Stand Plot Drill Jump Performance Cause Cause Drive Base Att Set Low Less Map Fun Things Head Head Trap File Report Dog Fun Place Trap Track Track Players Trap Drive Att Current Local Number Method Fun Jump Players Players D Dro Things Things Position Base Enemy Done Stand Cause Way Way Roll Enemy Performance Track Tracks Head Drive Reading Att Fun, Character Character Direction Trap Trap Trap Drive Kill D File Trap Roll Stand Spread Order Order Drill Peg Enemy Report Drill Spot Trap Feet Pie Rule Trap Trap Trap Diagn Rule Stand Fat Position Trap Guys Trap Trap Case Trap Case Ticket Tail Dro Feet Right Track Trap Peg Tracks Gems Trap Trap Trap Trap Trap Trap Tooth Roll Roll Trap Copy Trap Trap Base Column Spot Order Spread Spread Stand Foot Position Trap Drill Drill Drill Drill Drill Jump Dogs Trap Pack Trap Trap Trap Actions Tactics Drill Squ Fighter Spot Drill Field Stick Track Entry Order Number Drill Trap Trap Trap Trap Tail Trap Trap Trap Trap Enemy Neutral Guys Spot Peg Trap Trap Order Order Enemy Neutral Trap Trap Trap Trap Gems Trap Trap Trap Trap Trap Trap Trap Trap Drill Drill Drill Trap Trap Trap Trap Trap Trap Trap Monster Hitman Thing Door Trap Trap Trap Tactics Trap Trap Duel Trap Base Drill Trap Drill Drill Speed Trap Trap Trap Trap Trap Trap Drill Feet Drill Drill Trap Trap Trap Thing Trap Trap Trap Trap Trap Trap Trap Drill Drill Ground Structure Trap Guide Enemy Feet Number Column Monster Guys Duel Trap Trap Trap Trap Track Player Score Trap Trap Drill Drill Drill Drill Report Analy Frequency Squ Drill Drill Monster Trap Trap Deck Drill Ground Frequency Trap Trap Trap Copy Location Low Player Player Score Drill Head Actions Report Move Track Team File Trap Structure Number Slow Ratio Order Frequency Position Team Console Console Character Gems Trap Treasure Ratio Copy Physical Package File Door Drive Copy Way Physical Position Track Track Player Number Tail Command Package Base Command Tactics Table Copy Trap Base Location Spread File Trap Base Fine Fine Drill Ground Trap Drill Drill Work Size Order Location Tags Trap File Location Structure Fine Trap List List File Copy Tactics File Position Monster Drive Set Console Name Trap Slow Structure Trap Score Structure Structure Guide Squ Way Spread Character List Drill Position Enemy Number Mail Stand Each Drill Way List Fine Cause Enemy Location Done Fat Drill Drive Head Head Drive Trap Trap List Spread Location Spread Speed Done Mail Enemy Copy Copy Fine List Ground Drive Track Empty Order Copy Speed Work Copy Copy Order Drive Monster Set Size Track Head Tem Guide Ground Drive Report Trap Way Foot Location Structure Score Position Move Trap Team Tem File Trap Trap Slow Squ Tem Enemy Set Trap Drive Sign Salary Plot List Trap Less Salary Drive Level Source Track Copy Way Look Player Way Score Stand Location Done Done Drill Work Too Score Trap Trap Team Position Table Enemy Enemy Character Action Player Trap Squ Never Case Slow Head Tail Report Past Base Spread Report Tem Base Location Set Player Never Position Character Base Slow File Spread Drill Action Move Way List Work Stand Trap Moment Done Low Done Slow Tem Structure Structure Work Things Nothing Foot Move Track Position Source Stand Smart Drive Action Source Trap Action Way Squ Training Training Speed Enemy Smart File Team Speed Tem Track File Foot Stand Things Stand Spread Slow Trap Way Character Event Drive Actions Enemy Base Source Track Copy Actions File Event Things Location Actions Position Same Team Location Speed Fast Score Stand Speed Event Actions Player Player Players Drive Each Drive Order Location Less Players Nothing Action Actions Player Monster File Trap Work Done Player Less Fast Track Drill Track Ground Past Nothing Order Speed Fat Team Less Less Move Location Location Location Character Players Teams Others Each Door Door Stand Head Way Door Report Way Player Location Location Fast Drive Position Head Spread Door Actions Speed Drive Dim Trap Team Expl File Order Spread Order Way Character Trap Kill Location Location Trap Speed Foot Action Report Base Score Actions Report Move Things Fat Fine Action Way Others Way Together Player Action Each Console Structure Player Structure Fat Trap Trap Enemy None Stand Together Each Structure Structure Single Law Drive Team Spread Spread Team Spread Source Through Spread Fast Player Player Slow Position Character Things Move Actions Drive Fine Action File Player Trap Level Fast Fast Past File Console Drive Physical Structure Copy Order Physical Trap Base Head Source Trap Trap Smart Location Source Head None Single Location Speed Score Source Poor Things Nothing Poor Copy Speed Actions Way Players Copy Location Character Fast None Slow Smart Law Position Fine Ground File Slow Way Location