It helps if people state model,year and mileage in their post.

If your drums had worn out,can only assume it`s done very high mileage and is perhaps getting on a bit too.Presumably this means that several other brake components are past their best(read,unsafe)or have recently been replaced.

If you bled your brakes correctly and air bubbles were no longer present in the fluid exiting the rear bleed screws and you didn`t let the fluid drop too low in the reservoir then it`s either an incorrectly located component within the rear drums or a leaking wheel cylinder....in which case you`d see oil dripping out onto the wheel rim.

My money is on a damaged master cylinder with internal leak due to seal damage.This often happens on older,high-mileage vehicles because DIY mechanics,when bleeding the brakes,depress the pedal to the floor when they should only operate it within the range it had been working to when driving.You may have pushed the m/cyl piston seal into a corroded area and done the seals in.

You will bleed your brakes no quicker by depressing the pedal beyond its previous operating range but it may cost you a new master cylinder.