Destroying a droplet takes a bit of time as there are a few things happening behind the scenes. For instance, all long running droplets get automatic snapshots taken when they are destroyed so they can be recreated if needed. Also the entire disk area that contained your droplet gets written over to maintain the privacy of your data.
From the moment that you click destroy, you are no longer charged for the droplet. So there is no need to worry about additional costs.