If you want the system to lock a user after a set number of login attempts but are unsure how, this article explains the process.

Pluggable Authentication Module (PAM) includes pam_tally, a login counter module. It counts login attempts and resets the counter after a successful login. If attempts exceed the configured limit, it temporarily locks the user for a period you specify.

The file to configure is /etc/pam.d/system-auth.

Find the following line:

auth required pam_tally.so

Add deny=the number of permitted login attempts and unlock_time=the duration for which a user exceeding that limit (deny) is locked.

#%PAM-1.0

This file is auto-generated.

User changes will be destroyed the next time authconfig is run.

auth required pam_env.so
auth required pam_tally.so onerr=fail deny=3 unlock_time=300
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so

account required pam_unix.so
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account required pam_permit.so

password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass
use_authtok
password required pam_deny.so

session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond
quiet use_uid
session required pam_unix.so

If unlock_time is not set and a user exceeds the failed-login limit, the account remains locked. To make it usable again, run the following command.

#faillog -u username -r