Fetch pygame polys from ray manager.