Try doing the backwash to rinse a couple or three times before you return to filter mode. You obviously are not getting all the settled waste off the bottom of the filter tank.
Do you have a "sightglass" installed on your waste discharge line? If not you can do this very easily. Hayward makes them and they should be available at your local swimming pool supply store. The come in 1.5" and 2" sizes. Just cut the pipe where you want and glue it in. Then you can observe your waste water and you will KNOW when the water runs clear enough to return back to filter mode.
Mike