This may be because they have fled or died. It should be noted that keeping prisoners in castles or towns will result in them very, very gradually decrease in numbers, and eventually be gone altogether. You may want to avoid dragging around low value infantry, while keeping mounted prisoners around for a bit longer.

Prisoners are considered party members for the sake of calculating speed, thus the amount and speed of prisoners can result in increasing or decreasing Party Speed. Your prisoner management skill governs how many prisoners you can take at a time: 5 for every point. If you have a castle, you can check nearby towns until you find a broker, and then convey your prisoners to his location. If you have no town but have higher prisoner management, you can travel around capturing prisoners and selling them as you find ransom brokers. If you have a town, you can put all your prisoners in it until a ransom broker appears in your tavern. Prisoners can be a great source of money if you have high prisoner management, or a town or castle to store them in. In With Fire & Sword, sending your troops into battle without you will very often result in wounding enemy troops, which can be useful if on a quest to capture enemy soldiers. Money from ransoms can vary, depending how big the army was, if you captured a noble, or how good it is if it's a soldier (in classic Mount&Blade, all soldiers are worth 50 denars). Once you have taken prisoners, you may recruit them to your party, ransom them to Ransom Brokers (found in taverns in towns), or garrison them in a town or city. Very rarely, sharp weapons may result in wounding, due to reasons unknown (possibly the surgery skill on the opposite side). You can only capture the enemies who have been knocked unconscious through the use of blunt weapons ( horse charging is also blunt damage). Some fights do not allow the option, such as freeing a village from bandits.

You can take prisoners at the end of most battles or sieges.

Mount and blade keep getting captured