Is it safe to use lead fishing sinkers to weigh plastic plants down?

I have made my own artificial plants and to weigh them down I used hot glue to seal the outside of a lead fishing weight. If the glue eventually comes off or breaks a hole open in the seal, is it harmful to the fish?