morsepi Well, for the numbers you are posting, it seems you are far from something like a hard-limit, so I would look the problem in some other place.
There are several factors that can make that type of behavior, for start is good if you post your asterisk and elastix version, and also your codec of choice.
Also, when do you see this problem? when there are several concurrent calls? if so when it start? Does it happen all day long or an specific time? and lastly does your system reports some hard disk issues or large memory consumption?
Regards,
Claudio