The maximum team size dictates how many participants can join each team. You selected this when you created your challenge, but you can change it at any time. Here's how:
Open Your Admin Area
Log in to your admin area (your challenge URL with "/admin" at the end).
Open the Current Challenge's "Settings" Section
Click on "Teams"
Enter the New Maximum Team Size
Save this by clicking "Update" at the bottom of the page..