The system log entries are automatically purged and kept to the limit set in Setup > General Settings > Limit Activity Log, while client log entries will be kept indefinitely so the logs can easily reach many thousands.
I would suggest you navigate to Utilities > System > System Cleanup in your WHMCS admin area. Using this utility, you'll be able to clean up a lot of old activity, including the activity log.
Next, I would suggest cleaning up the tblactivitylog table by using SQL queries to remove some of the data. I would recommend starting with the following query:
DELETE FROM `tblactivitylog` WHERE date <= ( CURDATE() - INTERVAL 30 DAY );
That should delete all entries older than 30 days from the current date. You can change this to fit your needs, but please ensure you take a backup in case you need to undo any changes later.
