search

thunlp / LegalPLMs

Source code and checkpoints for legal pre-trained language models.
176 stars 25 forks source link

Use lawformer with transformers #3

Closed maicai closed 2 years ago

maicai commented 2 years ago

Hey！If I want to use lawformer with transformers to calculate case similarity, what should I do? could you share some demo?

Alice1222myy commented 2 years ago

Hey！If I want to use lawformer with transformers to calculate case similarity, what should I do? could you share some demo?

I have the same question with you. Have you got any solutions?

xcjthu commented 2 years ago

In the paper, we concate the query case and candidate case together, and feed the representation of [CLS] tokens to a MLP layer to calculate the similarity scores. By the way, as the global attention requires massive GPU memories, we can only train the model on V100 GPUs.

milktean commented 2 years ago

Hello ,have you solve the problem yet? I tried the model author provied,only get the output of Lawformer which is a LongformerBasemodeOutputWithPooling ,but I have no idea about how to finish the downstream tasks,such as legal judgement prediction

milktean commented 2 years ago

In the paper, we concate the query case and candidate case together, and feed the representation of [CLS] tokens to a MLP layer to calculate the similarity scores. By the way, as the global attention requires massive GPU memories, we can only train the model on V100 GPUs.

Can you share some demos so we can reproduce the results in the paper you publish.