Inspired by Sebastian Lague simulation* I decided to rework it for 3D.

The main idea is that each ant leave pheromones of “Home” or “Food” types, depending of where he goes from – that is, if ant goes from home/colony (magenta sphere) is leaves “Home” pheromones (magenta half-transparent cubes, and if it goes from food (solid blue spheres) it leave half-transparent blue boxes (“food” pheromones).

Each and consists of body – the only visible part represented with green capsule and two invisible parts: detector (big sphere of detection in front of it) and gland (interesting fact – ants’ organs, responsible for pheromones production are called “glands”) on the back to produce pheromones.

The logic of ant is simple as that:

detect is any “target” objects (food crops or colony) are in the Detector sphere. If so – cast the ray to check if it’s visible (and if not – ignore it)**. Choose closest object and go directly to it.
If no target was detected ant tries detects pheromones using algorithm similar to target detection, but instead of choosing the closest average vector for all detected pheromones will be calculated.
If pheromones of appropriate type are not detected too – it test if there is any “Wall” toward it, If wall detected – it turns around immediately.
If nothing was detected – it just goes forward slightly changing the direction from time to time.

On the beginning colony spawn 500 ants with target of “food”. When ant contacts the food crop it attaches (you can see they pulling small blue solid sphered back to colony) and turns around switching the target from “Food” to “Home”.

V1 simulation:
Ant produces pheromone every 500 steps, each pheromone stays for 15000 steps.

Some optimization to come:

• reduce the lifetime of pheromones as false paths to exhausted food source remains for too long (and to reduce the total number of object in scene)
• if the number of detected pheromones exceeds some limit – pause pheromone production for some time (to prevent too many pheromones around the colony)
  
  * https://www.youtube.com/watch?v=X-iSQQgOd1A&t=397s
  ** Only physical objects like walls and stones are considered