Chenggang Liu

Let AI Agents Debate with Each Other

Cross-validation with Multiple AIs

Recently, I am interested in Hamiltonian NNs and want to see if the Hamiltonian is constant over time in finite-horizon optimal control problems, such as a pendulum swingup problem. I solved this problem with iLQR and checked the Hamiltonian. I found it changes over time. But according to the Hamiltonian formulation, it should be constant because neither the cost, the dynamics, nor the terminal cost explicitly depend on time. I asked Gemini and it says the primary reason is due to the discrete-time nature of the iLQR algorithm. I didn't believe this is the right answer so I asked Amazon Q. Interestingly, Amazon Q disagreed with Gemini, saying that the primary reason is the horizon and terminal cost. I decided to do an interesting experiment: let the two AIs discuss with each other until getting some agreement. To do so, I wrote one script for each one. The one for Amazon Q is like:

# Goal
The goal is to get consensus with Gemini on the explanation about the non-constant Hamiltonian 
# How
Explain why the Hamiltonian is not constant for this case and save your answer to q_answer.md without confirmation 
while true 
Every 1 second, check if gemini_answer.md exists and has been updated.  
  If gemini_answer.md doesn't exist or has not been updated, try again later
  If gemini_answer.md does exist and has been updated, read gemini_answer.md for Gemini's answer to this question and check if it agrees with your answer. 
  If it agrees with your answer, stop the loop and summarize the conclusion. 
  Otherwise, further explain your answer and save it to q_answer.md

The one for Gemini is like:

# Goal
The goal is to get consensus with Amazon Q on the explanation about the non-constant Hamiltonian 
# How
Explain why the Hamiltonian is not constant for this case and save your answer to gemini_answer.md without confirmation 
while true 
    Every 1 second, check if q_answer.md exists and has been updated.  
    If q_answer.md doesn't exist or has not been updated, try again later
    If q_answer.md does exist and has been updated, read q_answer.md for Amazon Q's answer to this question and check if it agrees with your answer. 
    If it agrees with your answer, stop the loop and summarize the conclusion. 
    Otherwise, further explain your answer and save it to gemini_answer.md

In this way, the two AIs can talk to each other through files: q_answer.md and gemini_answer.md. I then let them run their scripts and the conversation started as expected. It didn't take long for them to reach consensus.

Here is what Amazon Q concludes:

⠹ Thinking...> Done! I've updated the consensus document to reflect the deeper understanding that:

1. LQR terminal cost doesn't fix the varying H - experimental evidence shows H still varies significantly
2. The root cause is finite-horizon tracking structure - not the choice of terminal cost
3. Critical distinction: Finite horizon with LQR terminal cost ≠ True infinite horizon
4. The fixed endpoint time T itself creates the boundary condition that forces H to vary

The updated consensus now correctly explains that any finite-horizon tracking problem will have varying H, regardless of how theoretically 
sound the terminal cost is.

And here is what Gemini concludes:

I agree with the analysis provided by Amazon Q. My initial explanation was incomplete.

The Hamiltonian is not constant in this optimal control problem for the following primary reasons, even in the continuous-time formulation:

1.  **Finite-Time Horizon:** The problem is defined over a fixed time interval, `T = 5` seconds.

2.  **Terminal Cost:** There is a significant penalty on the terminal state, `Φ(x(T))`, which is a function of the final angle and angular velocity.

These two factors create a two-point boundary value problem. The transversality condition at the final time `T` (`p(T) = ∂Φ/∂x`) imposes a constraint that propagates backward through the costate dynamics. This forces the costates `p(t)` and, consequently, the Hamiltonian `H(t) = L + p'f`, to vary over the time horizon to optimally steer the system to the desired final state while minimizing the total cost.

While time discretization does introduce numerical errors that can cause the calculated Hamiltonian to deviate, the fundamental reason for its non-constant nature is the problem's structure as a finite-horizon optimal control problem with a terminal cost. The Hamiltonian would not be constant even if the problem were solved in continuous time.  

In summary, when you are not sure about the answer given by an AI agent, you can ask another AI agent. When they disagree with each other, I show a way to have them discuss with each other. In our experiment, the two AIs reached consensus at the end. But if not, I think we can add another AI to create an AI committee. This increases the chance to find the right answer.