SEACrowd / seacrowd-datahub

A collaborative project to collect datasets in SEA languages, SEA regions, or SEA cultures.
Apache License 2.0
55 stars 54 forks source link

Create dataset loader for Automated Similarity Judgment Program (ASJP) #665

Open SamuelCahyawijaya opened 1 month ago

SamuelCahyawijaya commented 1 month ago

Dataloader name: asjp/asjp.py DataCatalogue: http://seacrowd.github.io/seacrowd-catalogue/card.html?asjp

Dataset asjp
Description The database of the Automated Similarity Judgment Program (ASJP) aims to contain 40-item word lists of all the world's languages. The database currently contains over 10,000 word lists rom over 5000 distinct languages.
Subsets -
Languages ran, khb, tyr, tlt, sqq, duq, pgu, tzn, mlf, dbe, blw, oni, ktv, xmz, cbn, sne, ebk, itx, oia, mqf, gbi, dms, kgq, mzt, pll, xxt, etz, tti, mgf, awr, llm, kiy, zbw, hmu, cwg, suc, khp, kzm, iwo, pne, hil, blf, bdw, btq, lbt, msl, kkb, wad, ssq, alo, tyz, kti, bti, bzp, vie, ssb, bna, kvr, sml, mxn, lbw, nev, ksx, pcb, tfo, mhe, spr, plc, phq, wnk, mnx, nec, vto, xsb, bkr, jah, sbo, sdo, bvk, kvw, bhz, bsb, prh, bps, tom, lpe, lbk, kja, toy, san, snv, ubl, ahh, sib, kyp, pku, rth, dmy, kkh, mhz, kyi, nun, tlv, kwr, duo, emb, slz, kwe, jmn, akb, tih, tlk, brv, awy, ttd, hbu, khm, kyn, tiy, ptu, blj, tmo, lbl, mso, rac, sou, wod, tmj, lnd, jbr, aoz, mhs, sxm, afz, awh, bto, ple, pag, law, spu, tcm, tjg, lha, mtg, tld, mgk, tmg, vkl, set, orz, bug, skz, hvn, ury, bqr, nbe, tha, kqe, abx, bsy, flh, szp, lus, and, mad, tpf, akl, eip, jav, agv, war, kzi, ilk, mvs, rmm, tft, tgq, ljp, ulm, rmh, msg, mbt, kge, kjr, cog, bed, ibl, ctd, due, kmk, kdw, vko, imr, amq, xkw, daw, woi, kxm, btx, amk, bnf, sun, wsa, ptt, prt, ges, iba, kjp, mbi, ilm, bny, ril, xau, aax, ayz, jet, tvm, snl, sda, twu, pse, plh, sza, nxx, ibg, kac, yac, knb, mdr, skv, sse, dni, kei, rka, yoy, tdj, kkv, sgu, ihp, alp, moo, kgw, nqm, tyn, had, btm, ilu, lti, dtp, max, krj, blx, mhy, saj, wtw, cts, pcc, jeh, kpm, kne, bnb, smk, lji, nlc, rnn, mvd, bku, bpp, bwp, pnp, aip, mng, tmu, lje, bls, nfa, baj, dei, syb, mwt, bqq, kag, ekg, wms, irh, ley, tlb, yka, wuy, twy, kyo, sti, twh, min, kli, anl, ivb, wgo, ren, tvw, bgl, dna, fbl, swt, ksc, tlg, wng, nkj, wfg, btn, swr, srl, kzv, xkq, acn, mxd, dgc, cmn, hnj, mqj, lmr, kvy, mkn, alk, mrh, bvz, dok, sge, grs, hal, mpy, kgv, tdi, irx, auu, tdd, rog, naa, isd, atz, ngt, mhx, bqb, tcz, bpg, bty, wrp, hpo, ddg, end, lbo, jra, txe, lex, pdu, dem, yue, bdl, ttn, llk, mzq, kje, kak, krr, nuf, sdu, ium, jei, lis, jak, kmt, dul, drn, nut, apx, ppk, lcc, wlw, bsa, klz, kpu, sxn, bkn, wai, cia, bcd, aqn, nxr, ksw, tnz, yki, agy, duw, kxd, cmr, trt, tby, mnw, kll, mro, atd, ror, mpz, mth, yir, blk, gei, sjm, zrs, plw, txn, att, rol, tds, tdf, ugo, kiq, yva, zbc, srw, szc, yli, bln, gop, sve, ssm, mrx, klw, bkx, mqn, asi, tam, emw, tmn, wrs, mwv, mfb, spi, dbn, lbn, kvo, lhn, txm, awv, dnw, kgi, pha, yog, kxf, xod, ivv, dun, bgk, thm, tes, jka, tty, szb, tlu, skb, rej, eng, smw, irr, ban, por, tbl, seu, wbb, lix, hni, mrf, ums, mog, mqr, tcq, krz, tsg, agf, mtj, mnz, dbj, asy, gal, brp, xte, wor, ceb, mba, laa, nbn, blz, tkx, kjm, ski, dnu, kyk, tdr, bnq, gdg, asl, yet, bkl, pmf, sbl, ljl, abl, kkl, kgr, skp, bts, ptn, lbx, pas, kbv, lcf, agt, tbk, ire, pwm, eky, asc, nst, tea, kvb, khe, ccp, bzi, akg, knq, wet, tml, zlm, sbg, lhu, dnt, wlo, kps, kts, ndx, msm, dro, hrk, tgn, prk, nod, mui, gad, xks, sgd, bay, tcg, atk, nsy, xdy, mbs, tni, bhc, opk, ifb, hnn, bcl, lur, mej, bno, mrw, lew, lnh, ree, bpr, kvz, krd, sob, khg, mkm, sea, tpu, aqm, kvd, brs, rob, jbj, tre, twg, cnh, kcd, spb, jax, mtd, lzn, auw, kuc, bzq, wmh, mok, duv, tpg, btw, ifu, zbe, bup, nrm, xwr, bya, cmo, bgv, shn, crw, bei, ktp, bgs, slg, mss, blr, kpq, kdy, mtq, dij, cje, myl, mkz, msk, eno, udj, tdn, sre, pac, ert, mqy, mrz, xxk, adn, aos, ahk, mmb, ace, tgb, kuv, laq, msb, mry, slu, mbb, rbl, drg, bnv, day, tts, cml, kwh, szw, kod, khh, beg, tmm, awu, cgc, atq, lcl, bwe, mak, ppm, pnx, pam, ktt, sko, bnd, snu, rbb, lev, tmw, gay, wow, abd, txx, mbd, ind, ner, kjg, tas, nxg, pru, nil, bbc, cja, nyl, loa, kbi, xmt, bdr, bfn, fau, mqt, zka, lra, xkl, grm, lao, tkd, slp, abs, lsi, lkj, mgm, nij, mya, xnn, oyb, mqs, kgb, pna, ila, mra, bhp, cjm, diy, jau, gor, beu, bzl, kjc, abp, zng, bpv, khc, mqx, knd, woo, lcp, uhn, lwh, btz, jhi, knx, air, rad, kpi, aol, kxn, scq, lmy, kta, mta, kwt, htu, phg, xse, slm, vme, bhw, jaq, brb, bpq, cma, cua, aws, sao, xmm, bdx, kem, frd, ilo, twe, wul, dbf, giq, kzz, tnm, uka, mmn, urk, bzu, scb, tvo, wno, sed, mxz, mel, pho, sow, duu, srv, bac, dmu, moq, kml, bfe, nia, tve, itv, pee, lwl, kyt, yno, itb, bks, obo, bjn, sas, sly, sbt, skh, psa, nxa, byl, gzn, huw, plv, kzu, nul, ddw, xbr, mnu, tgt, kvq, atb, abz, nks, kyu, obk, txt, nxl, jel, kns, itd, bru, kgx, mdh, raz, tnt, kzl, kly, kxq, kdt, jmd, blt, cyo, saw, sya, erw, otd, pmo, pce, pwo, zyp, heg, cps, xbn, wah, tnw, klg, ckh, dmr, bgi, yin, bgz, pkt, asz, ulf, tgl, sgb, mfa, tbw, mvx, msf, enr, psn, wbw, gir, akc, kqr, lvi, bhq, bsm, kzp, ifa, btd, pdo, puo, kzf, ify, wew, bdg, csy, txs, mky, raw, khd, smr, bdq, dsn, tbp, tad, nbq, cns, ret, tet, bgr, inn, mqo, nir, ccm, lgi, kht, vbb, bkd, mli, kig, btj, mnb, wbe, urn, kvu, wru, bth, lau
Tasks Word lists
License Creative Commons Attribution 4.0 (cc-by-4.0)
Homepage https://asjp.clld.org
HF URL -
Paper URL -