The four characters combine to paint a very literal and visceral picture: dragging mud and carrying water. This action is inherently inefficient, messy, and leaves a residue. This imagery is then applied to abstract situations, perfectly capturing the feeling of being bogged down by unnecessary details, words, or indecision.
This idiom is almost always used critically or as a form of negative advice. It's common in both formal and informal contexts.