search

jeya-maria-jose / UNeXt-pytorch

Official Pytorch Code base for "UNeXt: MLP-based Rapid Medical Image Segmentation Network", MICCAI 2022
https://jeya-maria-jose.github.io/UNext-web/
MIT License
459 stars 76 forks source link

Embedding dimension 768？ #10

Closed Capchenxi closed 1 year ago

Capchenxi commented 2 years ago

Hello, Quick question, I noticed that in the paper you said "In our experiments, we set H to 768 unless stated otherwise."

  1. Why 768? Is there any reason for this number? since 768 = 3256256 , I wonder if that number is relative to the shape of your input images or they have other special meanings?
  2. In the code you offer, there is an embedding vectors containing [32, 64, 128, 512], what is the relationship between the numbers in this vector and 768? Are these numbers the "stated otherwise"? Thanks!
dydxdt commented 2 years ago

Same question. 768 is inconsistent with the embed_dim(=[32, 64, 128, 512]) in the code. What's the relationship between them? I'm trying to train 3D medical images with this method and need to adjust these hyperparameters. Looking forward to the reply! Thanks!

jeya-maria-jose commented 1 year ago

The embedding dimension is 768 while tokenizing. Please refer this line: https://github.com/jeya-maria-jose/UNeXt-pytorch/blob/6ad0855114a35afbf81decf5dc912cd8de70476a/archs.py#L163