You can deactivate it instead. 

To deactivate your registry, log in and hover over the SETTINGS tab located at the top of the page. From the panel, select Profile & shipping info. At the bottom left of the page, click the link Deactivate MyRegistry Account.