To delete a custom block permanently, first make sure there are no copies of the block/script in the world or inside any other block. Then grab the block you want to delete and drag it, you should see a red X appear in the bottom right corner, drag the block there and it'll get deleted.