I buy a property on which the neighbor benefits from a

Yes. In principle, nothing prevents you from closing this building, unless there is a contrary clause in the constitutive act of servitude. If this is not the case, article 701 of the Civil Code prohibits the owner of the servient land from “reducing its use or making it more inconvenient”. Thus, if you decide to proceed with this closing, it will be necessary to give a key to your neighbor.