Technically, there's a difference between midgets and dwarfs.
Midgets have inproportional limbs to their torso, while dwarfs have normal proportions.
What you're talking about is dwarfs, an it only applies if you're a full grown adult, or close to being done with growth.
Up until about a year ago I was a dwarf. 4'11, 20 years old, and the doctors ran tests and concluded I was done growing, so legally I was a dwarf. But then shortly after I grew to about 5'1 in the span of 6 months. Stupid doctors.