Neural networks exhibit universal mapping capabilities that allow them to be used as a control tool for capturing hidden information about models learned from a dataset. The absence of an optimal statistical solution makes the featured neural network attractive despite the availability of complex and well-known filtering algorithms. In this work, we feature the theoretical aspects of a tracking algorithm based on neural network paradigm where, from discrete measurements provided by surveillance radar, the objective will be to estimate the target state for tracking purposes as accuracy as possible. The efficiency of the heuristic Q-Network method and effectiveness of the air combat maneuver strategy are verified by simulation experiments.ДЄir surveillance radar tracking systems present a variety of known problems related to uncertainty and lack of accurately in radar measurements used as source in these systems. Through continuous interaction with the environment, self-learning of the air combat maneuver strategy is realized. Aiming at the over-the-horizon air combat maneuver decision problem, the heuristic Q-Network method is adopted to train the neural network model in the over-the-horizon air combat training environment. At the same time, heuristic exploration and random exploration are combined.

In order to improve the efficiency of the reinforcement learning algorithm for the exploration of strategy space, this paper proposes a heuristic Q-Network method that integrates expert experience, and uses expert experience as a heuristic signal to guide the search process. Based on the characteristics of over-the-horizon air combat, this paper constructs a super-horizon air combat training environment, which includes aircraft model modeling, air combat scene design, enemy aircraft strategy design, and reward and punishment signal design. With the development of information technology, the degree of intelligence in air combat is increasing, and the demand for automated intelligent decision-making systems is becoming more intense. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Then, a phased training method, called 'basic-confrontation', which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN).

A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats.

Acm air combat maneuvers