Referees determine fouls during games based on their interpretation of player actions, adherence to the established rules, and the context of the play. They assess physical contact, obstruction, offside positions, and any unsportsmanlike behavior that may lead to rule violations. Referees must maintain a clear perspective on the field, often making instantaneous decisions while relying on their training and experience to ensure fairness and uphold sportsmanship.