The branch change policy of IIT Mandi is based on the CGPA you scored in the first and second semester examinations.
There is nothing to do with caste in the selection process for branch change policies. Make sure you have scored a minimum of 8.5 CGPA in both semesters.
The competition level is increasing every year as most students wish to switch to the CSE branch since it opens a wide array of opportunities. The branch change would be easier for you if you strived hard and consistently followed a daily study routine.