search

NVSL / linux-nova

NOVA is a log-structured file system designed for byte-addressable non-volatile memories, developed at the University of California, San Diego.
http://nvsl.ucsd.edu/index.php?path=projects/nova
Other
422 stars 117 forks source link

Snapshots do not work on DAX mmap()'d files without commenting out lines mm/memory.c:1549-1552 #107

Open derrickgreenspan opened 3 years ago

derrickgreenspan commented 3 years ago

When attempting to take snapshots on a filesystem with mmapped files, the kernel will oops and future snapshots will never complete. The kernel generates a warning at mm/memory.c:1549 immediately before the oops.

This is the offending code:

if (pte_pfn(*pte) != pfn_t_to_pfn(pfn)) {
     WARN_ON_ONCE(!is_zero_pfn(pte_pfn(*pte)));
     goto out_unlock;         
}

Commenting this code out allows snapshots to proceed without the kernel crashing. I'm not sure whether the resulting snapshot is valid; given that snapshots worked in the past, and earlier kernels did not have this specific code, I suspect that the issue is a regression caused by major changes to the Linux kernel memory management system since NOVA was introduced.

derrickgreenspan commented 3 years ago

I now believe I understand the consequences of commenting that code out: snapshots will destroy mmap()'d DAX data. Commenting out entry=pte_mkyoung(*pte) and entry=maybe_mkwrite(pte_mkdirty(entry, vma); fixes that.

Andiry commented 3 years ago

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

derrickgreenspan commented 3 years ago

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

Yes, this fix is a workaround, not part of NOVA code.

I also wanted to ask: the performance of taking snapshots depends on the number of previous snapshots (declining slowly), even if old snapshots are deleted. Is this expected behavior?

Andiry commented 3 years ago

That is not expected. Snapshot taking time should not increase with the number of snapshots taken.

On Tue, Jun 29, 2021 at 1:47 PM Derrick Greenspan @.***> wrote:

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

Yes, this fix is a workaround not part of NOVA code.

I also wanted to ask: the performance of taking snapshots depends on the number of previous snapshots, even if old snapshots are deleted. Is this expected behavior?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870906024&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=BtmOJFBbE_9I9EYA3uuo4OM0Sg6zCyRvqrOArZ6ejgg&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEG62D5W7XL5OQ5JX5LTVIWPJANCNFSM47K7ALYA&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=DY-oq0qiWpOPoExItWsrZUa7UkWYJouATIYxWi_8BXw&e= .

derrickgreenspan commented 3 years ago

Hmm. Is there any way you could try taking snapshots of mmap'd() data yourself and see if it works without modification? Has NOVA been used this way on actual persistent memory such as Optane?

Best,

Derrick Greenspan MSCS PhD Graduate Research Assistant

From: Andiry Xu @.> Sent: Tuesday, June 29, 2021 7:36 PM To: NVSL/linux-nova @.> Cc: Derrick.Greenspan @.>; Author @.> Subject: Re: [NVSL/linux-nova] Snapshots do not work on DAX mmap()'d files without commenting out lines mm/memory.c:1549-1552 (#107)

That is not expected. Snapshot taking time should not increase with the number of snapshots taken.

On Tue, Jun 29, 2021 at 1:47 PM Derrick Greenspan @.***> wrote:

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

Yes, this fix is a workaround not part of NOVA code.

I also wanted to ask: the performance of taking snapshots depends on the number of previous snapshots, even if old snapshots are deleted. Is this expected behavior?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870906024&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=BtmOJFBbE_9I9EYA3uuo4OM0Sg6zCyRvqrOArZ6ejgg&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEG62D5W7XL5OQ5JX5LTVIWPJANCNFSM47K7ALYA&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=DY-oq0qiWpOPoExItWsrZUa7UkWYJouATIYxWi_8BXw&e= .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNVSL%2Flinux-nova%2Fissues%2F107%23issuecomment-870986800&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876933432%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v8pty0aIQk1V5N3E0o1XqPs46DSczS7xBos8REiUOps%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADMHKXWL6IMDU436K4KM2O3TVJKHPANCNFSM47K7ALYA&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876943427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M4um3VHx61sbQdt3Q6AdFrt6Uh0Td4GWpjEqGBQLofg%3D&reserved=0.

Andiry commented 3 years ago

Not sure about that. I have graduated for quite some time and am not working on NOVA anymore... Also I don't have access to Optane memory.

On Tue, Jun 29, 2021 at 4:39 PM Derrick Greenspan @.***> wrote:

Hmm. Is there any way you could try taking snapshots of mmap'd() data yourself and see if it works without modification? Has NOVA been used this way on actual persistent memory such as Optane?

Best,

Derrick Greenspan MSCS PhD Graduate Research Assistant

From: Andiry Xu @.> Sent: Tuesday, June 29, 2021 7:36 PM To: NVSL/linux-nova @.> Cc: Derrick.Greenspan @.>; Author @.> Subject: Re: [NVSL/linux-nova] Snapshots do not work on DAX mmap()'d files without commenting out lines mm/memory.c:1549-1552 (#107)

That is not expected. Snapshot taking time should not increase with the number of snapshots taken.

On Tue, Jun 29, 2021 at 1:47 PM Derrick Greenspan @.***> wrote:

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

Yes, this fix is a workaround not part of NOVA code.

I also wanted to ask: the performance of taking snapshots depends on the number of previous snapshots, even if old snapshots are deleted. Is this expected behavior?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870906024&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=BtmOJFBbE_9I9EYA3uuo4OM0Sg6zCyRvqrOArZ6ejgg&e=%3E , or unsubscribe < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEG62D5W7XL5OQ5JX5LTVIWPJANCNFSM47K7ALYA&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=DY-oq0qiWpOPoExItWsrZUa7UkWYJouATIYxWi_8BXw&e=%3E .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNVSL%2Flinux-nova%2Fissues%2F107%23issuecomment-870986800&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876933432%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v8pty0aIQk1V5N3E0o1XqPs46DSczS7xBos8REiUOps%3D&reserved=0> https://urldefense.proofpoint.com/v2/url?u=https-3A__nam02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252FNVSL-252Flinux-2Dnova-252Fissues-252F107-2523issuecomment-2D870986800-26data-3D04-257C01-257Cderrick.greenspan-2540knights.ucf.edu-257Cbe4906bba73c4ee5d3f508d93b56b4c2-257C5b16e18278b3412c919668342689eeb7-257C0-257C0-257C637606065876933432-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26sdata-3Dv8pty0aIQk1V5N3E0o1XqPs46DSczS7xBos8REiUOps-253D-26reserved-3D0-253E&d=DwQFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=QVGLl25NjCiYMGn1lI30Ou3sZPqaQqS_KUPoQNku6wE&e=, or unsubscribe< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADMHKXWL6IMDU436K4KM2O3TVJKHPANCNFSM47K7ALYA&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876943427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M4um3VHx61sbQdt3Q6AdFrt6Uh0Td4GWpjEqGBQLofg%3D&reserved=0> https://urldefense.proofpoint.com/v2/url?u=https-3A__nam02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Fnotifications-252Funsubscribe-2Dauth-252FADMHKXWL6IMDU436K4KM2O3TVJKHPANCNFSM47K7ALYA-26data-3D04-257C01-257Cderrick.greenspan-2540knights.ucf.edu-257Cbe4906bba73c4ee5d3f508d93b56b4c2-257C5b16e18278b3412c919668342689eeb7-257C0-257C0-257C637606065876943427-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26sdata-3DM4um3VHx61sbQdt3Q6AdFrt6Uh0Td4GWpjEqGBQLofg-253D-26reserved-3D0-253E&d=DwQFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=fSxrbRR5qfa6DtYIS-T5Tnb6KzLxwmtfkpimuYlfYhA&e= .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870988002&d=DwMFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=jc6rhLx_EGBwy72xNOmEnzLQDZ3GzFptIxmbZz7GEZk&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEEIQL6NRX3FHDCKTOTTVJKS3ANCNFSM47K7ALYA&d=DwMFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=eS3z_RGqzbozfqOO8g2hCRiTV3bdo9MaJweTG72DV68&e= .

derrickgreenspan commented 3 years ago

Ok, are there other maintaners working on NOVA who I could contact?

Thank you!

Derrick Greenspan MSCS PhD Graduate Research Assistant

From: Andiry Xu @.> Sent: Tuesday, June 29, 2021 7:42 PM To: NVSL/linux-nova @.> Cc: Derrick.Greenspan @.>; Author @.> Subject: Re: [NVSL/linux-nova] Snapshots do not work on DAX mmap()'d files without commenting out lines mm/memory.c:1549-1552 (#107)

Not sure about that. I have graduated for quite some time and am not working on NOVA anymore... Also I don't have access to Optane memory.

On Tue, Jun 29, 2021 at 4:39 PM Derrick Greenspan @.***> wrote:

Hmm. Is there any way you could try taking snapshots of mmap'd() data yourself and see if it works without modification? Has NOVA been used this way on actual persistent memory such as Optane?

Best,

Derrick Greenspan MSCS PhD Graduate Research Assistant

From: Andiry Xu @.> Sent: Tuesday, June 29, 2021 7:36 PM To: NVSL/linux-nova @.> Cc: Derrick.Greenspan @.>; Author @.> Subject: Re: [NVSL/linux-nova] Snapshots do not work on DAX mmap()'d files without commenting out lines mm/memory.c:1549-1552 (#107)

That is not expected. Snapshot taking time should not increase with the number of snapshots taken.

On Tue, Jun 29, 2021 at 1:47 PM Derrick Greenspan @.***> wrote:

Thanks for the findings. I guess the fix you suggested is not part of NOVA, right? So it is a workround?

Yes, this fix is a workaround not part of NOVA code.

I also wanted to ask: the performance of taking snapshots depends on the number of previous snapshots, even if old snapshots are deleted. Is this expected behavior?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870906024&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=BtmOJFBbE_9I9EYA3uuo4OM0Sg6zCyRvqrOArZ6ejgg&e=%3E , or unsubscribe < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEG62D5W7XL5OQ5JX5LTVIWPJANCNFSM47K7ALYA&d=DwMCaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=W0urqJnKjjEtHkZ-8SrpVpS_DAh1sqkeZpQhuxz5rPA&s=DY-oq0qiWpOPoExItWsrZUa7UkWYJouATIYxWi_8BXw&e=%3E .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNVSL%2Flinux-nova%2Fissues%2F107%23issuecomment-870986800&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876933432%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v8pty0aIQk1V5N3E0o1XqPs46DSczS7xBos8REiUOps%3D&reserved=0> https://urldefense.proofpoint.com/v2/url?u=https-3A__nam02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252FNVSL-252Flinux-2Dnova-252Fissues-252F107-2523issuecomment-2D870986800-26data-3D04-257C01-257Cderrick.greenspan-2540knights.ucf.edu-257Cbe4906bba73c4ee5d3f508d93b56b4c2-257C5b16e18278b3412c919668342689eeb7-257C0-257C0-257C637606065876933432-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26sdata-3Dv8pty0aIQk1V5N3E0o1XqPs46DSczS7xBos8REiUOps-253D-26reserved-3D0-253E&d=DwQFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=QVGLl25NjCiYMGn1lI30Ou3sZPqaQqS_KUPoQNku6wE&e=, or unsubscribe< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADMHKXWL6IMDU436K4KM2O3TVJKHPANCNFSM47K7ALYA&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Cbe4906bba73c4ee5d3f508d93b56b4c2%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606065876943427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=M4um3VHx61sbQdt3Q6AdFrt6Uh0Td4GWpjEqGBQLofg%3D&reserved=0> https://urldefense.proofpoint.com/v2/url?u=https-3A__nam02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Fnotifications-252Funsubscribe-2Dauth-252FADMHKXWL6IMDU436K4KM2O3TVJKHPANCNFSM47K7ALYA-26data-3D04-257C01-257Cderrick.greenspan-2540knights.ucf.edu-257Cbe4906bba73c4ee5d3f508d93b56b4c2-257C5b16e18278b3412c919668342689eeb7-257C0-257C0-257C637606065876943427-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C1000-26sdata-3DM4um3VHx61sbQdt3Q6AdFrt6Uh0Td4GWpjEqGBQLofg-253D-26reserved-3D0-253E&d=DwQFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=fSxrbRR5qfa6DtYIS-T5Tnb6KzLxwmtfkpimuYlfYhA&e= .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NVSL_linux-2Dnova_issues_107-23issuecomment-2D870988002&d=DwMFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=jc6rhLx_EGBwy72xNOmEnzLQDZ3GzFptIxmbZz7GEZk&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAKBYEEIQL6NRX3FHDCKTOTTVJKS3ANCNFSM47K7ALYA&d=DwMFaQ&c=-35OiAkTchMrZOngvJPOeA&r=HMwrvpNIfGkJhfYgtfzIcE0EiyctTLdPd2f9rzvJviU&m=haULV5D-ivW8Z-gdOHfNFu73kdAhG6zZfXmHPEhYE6k&s=eS3z_RGqzbozfqOO8g2hCRiTV3bdo9MaJweTG72DV68&e= .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNVSL%2Flinux-nova%2Fissues%2F107%23issuecomment-870989065&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Ce373478455034bcd5a3e08d93b578cfc%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606069476005351%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bm1VVGn5ukbkHo91X1RKIKOVXWnxnDNgtfAYG8z9GSw%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADMHKXTY7GKE6KEPIDGWIZ3TVJK6DANCNFSM47K7ALYA&data=04%7C01%7Cderrick.greenspan%40knights.ucf.edu%7Ce373478455034bcd5a3e08d93b578cfc%7C5b16e18278b3412c919668342689eeb7%7C0%7C0%7C637606069476015344%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RzScqgt4VB%2BRHg0ibu4yCqafzT19NLzvJ6pXWxuFnV8%3D&reserved=0.