I tend to assume the area of the pentagram spell is the circle on the outside, and that the demon will be summoned into the central pentagon. Demon will be forced to fit into that size, whatever it is. (Assuming the pentagram spell can hold it at all.)

I agree that it is stupid not to indicate (1) what part of the pentagram is used to calculate the area and (2) if that's the circle, basing the cost on radius. Hell, if that's not the circle, the cost should still be based on radius or some comparable measurement (center to point of star; center to point of pentagon, whatever) because "10 sq feet" is a ridiculous number for anything other than a square, and, as per the title of the spell, it's not going to be square when it's done.