If possible use the warm packs to heat up the nesting materials and fur before placing the babies in the nestbox. Substitutes for a rice bag can be an electric heating pad underneath the nestbox or place the nestbox over or near a heat register in the house. Lay a 2 liter bottle filled with hot water in the nestbox and cover with a towel to heat up the box without the babies in it. Or try an old fashioned water bottle underneath or in the nestbox covered with a towel over the water bottle.
If you are using the rice bags or other heating method to try to warm up the babies and not just the box, be careful. Put babies on top of cloth but only when it is determined not to be too hot for them. Remember they cannot regulate their body heat, their skin is much thinner and sensitive than yours. So while it might feel warm and cozy to you, it could burn or overheat them. They are more likely to succumb to the heat than be able to move away from it. Check on babies often to make sure they aren't getting too hot and don't heat them up too quickly. The shock of the latter can kill them as easily as the cold does.
I've also heard of folks placing the babies in an open top baggie and floating them in some warm water. This should be done under a watchful eye and again not too hot water. Nor let the water get too cool to chill them.
Personally the safest and best method I've found is using your own body heat, it is more gradual and regulated. You can hold them in your hands to warm them up but I've found it creates some moisture on them which will wick away body heat as the moisture dissipates. So put them inside your shirt or next to your skin and they should be squirming-vermin warm in no time.