Automate your Computer with python
skip to ‘‘Getting Started’’ section if you don’t want to know about module used and it’s installation
Hello Readers, I hope you are good. Today you will learn “How can you automate your Computer Mouse and Keyboard using Python”. That’s why i love Python, it has every functionality i want. Entering to Python World, solved my Many Problems and Queries that i had on my mind. One thing, when i try to learn something i just Google it and Boom i get some articles or Helpful tips to make my own way to do that thing. In this article, we will be using a Python module PyAutoGUI that automates our mouse and keyboard. I saw many articles related to PyAutoGUI but here i will try to give you some unique ideas of using this module.
The purpose of PyAutoGUI is to provide a cross-platform Python module for GUI automation for human beings. The API is designed to be as simple as possible with sensible defaults. I will touch only the basics of this module, the rest you can look for further attributes and features in the documentation of the PyAutoGUI module at PyPI.
Installation:
Windows:
On Windows, it is much easier to install and use the PyAutoGUI. On windows, you can install it from the PyPI using pip if you have pip installed.
pip install pyautoguiOs X:
On Os X, you need to install some extra modules to be able to run PyAutoGui.
pip3 install pyobjc-core
pip3 install pyobjc
pip3 install pyautoguiLinux:
On Linux, You have to install some packages.
pip3 install python3-xlib
sudo apt-get install scrot
sudo apt-get install python3-tk
sudo apt-get install python3-dev
pip3 install pyautoguiGetting Started:
If you want to make an incredible Automated Tool, you can integrate this module into other awesome modules like Selenium, OpenCV etc.
You can do the following things using PyAutoGUI.
Fail-Safes:
Mouse Features:
- Mouse Movement (Relative, Fixed)
- Dragging (Relative, Fixed)
- Clicks (Right, middle, Left, up, down, Double and Triple)
- Scrolling (up, down, left and right)
Keyboard Features:
- Writing
- Pressing keys (f1 — f2, all other keys)
- Press Shortcut (e.g. CTR+C)
Message Box Functions:
Screenshot Functions (Image Search):
Other Features:
Fail-Safes:
Pause:
Pause is an option in PyAutoGUI that sets the pause between each call of the module. For Example: we have two functions one write and other clicks, the second function is executed after configured duration when set this option.
pyautogui.PAUSE = dur # 2.5Fail safe:
This option is more works like canceling the program using CTRL + C. If this option is set to True , it aborts program and throws a pyautogui.FailSafeException Exception by moving mouse to upper-left corner of the screen.
pyautogui.FAILSAFE = TrueMouse Features:
Mouse Movement:
You can Move mouse to x,y coordinate of the screen using the following function of the pyautogui class
pyautogui.moveTo(x, y, duration=num_seconds)where x and y are XY coordinates on the screen. X increases going right, Y increases going down. duration is the time in seconds that makes the movement of the mouse slow according to time.
There is an another Function that move the mouse relative to the current position of the mouse, which simply means, it will move mouse (100,100) to left and right from where the current mouse position is.
pyautogui.moveRel(xOffset, yOffset, duration=num_seconds)# you can use pyautogui.move(xOffset, yOffset) as well
Here, the xOffset and yOffset are the increments to current position of the mouse x and y respectively.
Dragging:
Dragging is another awesome and important feature of the mouse, Dragging simply means move the mouse while holding the click button. You can do dragging by the following function.
pyautogui.dragTo(x, y, duration=num_seconds)pyautogui.dragRel(xOffset, yOffset, duration=num_seconds)# you can use pyautogui.drag(xOffset, yOffset) as well
The explanation for these functions is the same as mouse movement.
Clicks:
As you know, there are different types of clicks we can perform using our mouse manually. PyAutoGUI can let us use many of that features. Some of them are:
- Single, Double and Triple Click
- Mouse Button Up and Down Event
- Left, Right and Middle Click
Syntax:
pyautogui.click(x=None, y=None, clicks=1, interval=0.0, button='left', duration=0.0)You can use click() function like this:
pyautogui.click() # click the mouse at current positionpyautogui.click(20,10) # move to 20, 10, then click the left mouse button# you can pass a Tuple as well (20,10)pyautogui.click(button='right') # right-click the mousepyautogui.click(button='middle') # middle-click the mousepyautogui.click(button='left') # left-click the mouse# You can pass it with the above as wellpyautogui.click(clicks=2) # double-click the left mouse buttonpyautogui.click(clicks=2, interval=0.25) # double-click the left mouse button, but with a quarter second pause in between clickspyautogui.click(button='right', clicks=3, interval=0.25) ## triple-click the right mouse button with a quarter second pause in between clicks
there are separate functions as well to get the same work.
pyautogui.doubleClick()# perform a left-button double clickpyautogui.rightClick()# perform a right-button clickpyautogui.middleClick()# perform a middle-button clickpyautogui.trippleClick()# perform a left-button triple clickHave more Optional Attributes for each Functions such as x and y coordinates, duration, and interval.
Mouse clicks and drags are composed of both pressing the mouse button down and releasing it back up. If you want to perform these actions separately, call the mouseDown() and mouseUp()functions. They have the same x, y, and button .
pyautogui.mouseDown(); pyautogui.mouseUp() # does the same thing as a left-button mouse clickpyautogui.mouseDown(button='right') # press the right button downpyautogui.mouseUp(button='right', x=100, y=200) # move the mouse to 100, 200, then release the right button up.
Scrolling:
The mouse scroll wheel can be simulated by calling the scroll() function and passing an integer number of “clicks” to scroll. The amount of scrolling in a “click” varies between platforms.
Syntax:
pyautogui.scroll(clicks, x=None, y=None)You can use scroll() function like this:
pyautogui.scroll(10) # scroll up 10 "clicks"
pyautogui.scroll(-10) # scroll down 10 "clicks"
pyautogui.scroll(10, x=100, y=100) # move mouse cursor to 100, 200, then scroll up 10 "clicks"You can use hscroll() for scrolling left and right in linux and OSX platform.
Keyboard Features:
Writing:
You can write an alphabet, word or sentence with the help of a function in the pyautogui class. It also offers the delay between each typing word which makes it more fantastic.
pyautogui.typewrite('Hello world!') # prints out "Hello world!" instantlyIf you want to animate this sentence and want computer to write it after a small period then you can use interval attribute.
pyautogui.typewrite('Hello world!', interval=0.25)
# prints out "Hello world!" with a quarter second delay after each characterPressing keys:
To press these keys, call the press() function and pass the defined values in string to the function.
To press more than one key, pass a tuple or list of keys, it also take another optional attribute presses=1 that limits the number of presses of the key(s).
There is also one more attribute interval=0.0 which is interval between each key press.
Keys for this function is defined and can be retrieved by the following option pyautogui.KEYBOARD_KEYS . It will print a list of defined keys it support. For example: enter, up, down, esc, f1. See KEYBOARD_KEYS.
pyautogui.press('enter') # press the Enter keypyautogui.press('f1') # press the F1 keypyautogui.press('left') # press the left arrow key
press() is just wrapper forkeyUp() and keyDown() functions, which simulate pressing a key down and then releasing it up.
pyautogui.keyDown('shift') # hold down the shift keypyautogui.keyUp('shift') # release the shift key
Pressing Shortcuts:
To make pressing hotkeys or keyboard shortcuts convenient, the hotkey() can be passed several key strings which will be pressed down in order, and then released in reverse order.
pyautogui.hotkey('ctrl', 'shift', 'esc') # press CTRL + SHIFT + ESC at the same timeThe above code is equivalent to
pyautogui.keyDown('ctrl') # Hold CTRL button
pyautogui.keyDown('shift') # Hold SHIFT button
pyautogui.keyDown('esc') # Hold ESC button
pyautogui.keyUp('esc') # Release ESC button
pyautogui.keyUp('shift') # Release SHIFT button
pyautogui.keyUp('ctrl') # Release CTRL buttonMessage Box Functions:
PyAutoGUI also display JavaScript-style message boxes. There are four message box functions provided:
Alert Window:
Displays a simple message box with text and a single OK button. Returns the text of the button clicked on.
pyautogui.alert(text='', title='', button='OK')Confirm Window:
Displays a message box with OK and Cancel buttons. Number and text of buttons can be customized. Returns the text of the button clicked on.
pyautogui.confirm(text='', title='', buttons=['OK', 'Cancel'])Prompt Window:
Displays a message box with text input, and OK & Cancel buttons. Returns the text entered, or None if Cancel was clicked.
pyautogui.prompt(text='', title='' , default='')Password Window:
Displays a message box with text input, and OK & Cancel buttons. Typed characters appear as *. Returns the text entered, or None if Cancel was clicked.
password(text='', title='', default='', mask='*')Screenshot Functions (Image Search):
PyAutoGUI can take screenshots, save them to files, and locate images within the screen. This is useful if you have a small image of, say, a button that needs to be clicked and want to locate it on the screen
Take a Screenshot:
Calling screenshot() will return an Image object. Passing a string of a filename will save the screenshot to a file as well as return it as an Image object.
im1 = pyautogui.screenshot()
im2 = pyautogui.screenshot('my_screenshot.png')There is also an optional region keyword argument, if you do not want a screenshot of the entire screen. You can pass a four-integer tuple of the left, top, width, and height of the region to capture:
im = pyautogui.screenshot(region=(0,0, 300, 400))Locate Image on Screen:
You can visually locate something on the screen if you have an image file of it. For example, say the calculator app was running on your computer and looked like this:
and you have an image of the button, such as the image of the 7 button:
you can call the locateOnScreen('7key.png') function to get the screen coordinates. The return value is a 4-integer tuple: (left, top, width, height). However if you just want the x and y coordinates of the image on the screen you can call locateCenterOnScreen() which will return only x and y coordinates. If the image on the screen doesn’t found, it will raise ImageNotFoundException . If you want to click that image or object, just pass the x and y coordinates from the return values of the previous functions to click() function. To know more about Locating objects on the screen visit this link.
Other Features:
Screen Size:
To get the Screen Size, PyAutoGUI has a function size() that returns tuple of integers: (width, height) for size of the main monitor.
pyautogui.size() # output: Size(width=1366, height=768)Position of the Cursor:
To get the current position of the cursor, PyAutoGUI has a function position() that returns tuple of integers: (x, y) for the current position of the mouse cursor.
pyautogui.position() # outpu: Point(x=487, y=664)Credits:
- Images are taken from Google Images.
- Thanks to Al Sweigart for the awesome Module.
📝 Read this story later in Journal.
👩💻 Wake up every Sunday morning to the week’s most noteworthy stories in Tech waiting in your inbox. Read the Noteworthy in Tech newsletter.
