In a word, YES.
As well as being unsightly, leaving a fused connection unit (FCU) or switched fused connection unit with out the cover in place will be potentially dangerous.
The IP (International Protection) will also be compromised, allowing the easy ingress of objects (kitchen food stuffs and liquids) as well as the potential for coming in contact with live parts and leading to electric shock.
Refer to Regulation 512.2.1 of the 18th edition of the wiring regulations BS7671 & BS 1363-4:2016+A1:2018.
Leaving the fuse out would also mean the equipment that is attached to the FCU would not function (as well as the dangers mentioned above).
The unit should be replaced by a qualified electrician and the correct rated fuse installed suitable for the equipment and wiring that is is protecting.
Selecting the correctly rated fuse is very important. Most equipment will either be 3amps or 13amps, (5 & 10amp fuses are still common in the plugs of some IT equipment). For example if the FCU is protecting the power feeding a boiler then it will likely be a 3 amp fuse. If it is an FCU for a washing machine then probably a 13amp fuse. The manufacturer’s instructions should always be followed and using a qualified electrician to do the replacement will ensure it is done correctly and safely. Get it wrong and it could lead to electric shock, fire or even death.