What technology does your Application under Test use?
That's the main question when searching for a library to steer your application.
Sure, there are libraries out there which use image and text recognition to find controls and interact with them.
But nothing beats a library which can steer your application natively via its exposed interfaces.
Web Applications
Libraries steering the UI of a Web Application in a Web Browser
There are two main libraries which can automate web applications in a web browser:
Libraries that can automate Web Services and RESTful APIs
Desktop Applications
Does it offer a User Interface?
Automation via the User Interface is a common approach, even though it can be very unstable, slow and heavy on maintenance.
But if your Application offers a User Interface, there is also the possibility to automate it via that User Interface.
That's where the technology used becomes important.
General Windows Desktop UI Libraries
There are libraries out there for general Windows desktop automation or image-based automation.
But if you know that it's written in Java - check out some SWING or SWT Libraries!
Java Specific UI Libraries
Some Libraries to steer the User Interface of your Java Application