Welcome once again to my blog folks!! In this post we will learn how we can tackle basic problems related to memory issues and more by using some basic groovy scripts on our AEM server.
Below are some scenarios where we can use groovy scripts to make our life easy in AEM: 1. List of unused Assets in DAM. 2. List of unused Components. 3. Modify properties. 4. List of heavy content in AEM repo.
We can implement groovy scripts using two way, one is using the already built groovy console and secondly we can use API calls to perform the operations.
1. Groovy Console:
Go to Groovy package and clone the latest version (14.0.0) in your local and build the code using mvn install -P local. To verify the installation go to Groovy Console. After verification run the below mentioned script to get unused asset and non referenced Asset in DAM.
Groovy Console in localhost
/*
Assets which are not referenced in the content and could be removed.
@author Nikhil Kumar */
def predicates = [path: “/content/dam/we-retail”, type: “dam:Asset”, “orderby.index”: “true”, “orderby.sort”: “desc”]
def query = createQuery(predicates)
query.hitsPerPage = 100
def result = query.result
println “${result.totalMatches} hits, execution time = ${result.executionTime}s\n–”
result.hits.each { hit ->
def path=hit.node.path
Resource res = resourceResolver.getResource(path)
if(res!=null){
getAllReferences(path);
}
}
def getAllReferences(assetpath) {
def queryManager = session.workspace.queryManager
def queryStatement = “/jcr:root” + “/content/we-retail//*” + “[jcr:contains(., ‘”+assetpath+”‘)]”
def query = queryManager.createQuery(queryStatement, “xpath”)
def result = query.execute()
def rows = result.getRows().size
if(rows==0){
println “Unused Asset: “+assetpath
}
}
Result in groovy console
2. API Calls to Groovy server:
Scripts can be remotely executed by sending a POST request to the console servlet with either the scriptPath or scriptPaths query parameter.
Single Script :
curl -d "scriptPath=/var/groovyconsole/scripts/samples/testG.groovy" -X POST -u admin:admin http://localhost:4502/bin/groovyconsole/post.json
Multiple Scripts:
curl -d "scriptPaths=/var/groovyconsole/scripts/samples/JcrSearch.groovy&scriptPaths=/var/groovyconsole/scripts/samples/testG.groovy" -X POST -u admin:admin http://localhost:4502/bin/groovyconsole/post.json
Groovy scripts are saved at the location /var/groovyconsole/scripts/samples/.. in AEM 6.5.5 and many scripts can be added as per the requirement.
Groovy scripts path in CRX
Now let’s understand how we can run groovy scripts on the AEM server using Postman. Create a new collection and create a new request in it (I am using the same collection of Elastic search), with following settings and do a POST call.
AEM Authentication Config.Providing scriptPath
Once we are done with the required settings in Postman, now we will do a POST call and see the response. Below is the response I got when I executed groovy script on we-retail DAM.
API response
We can implement API approach to build our own AEM tool to run different scripts and have our own dashboard in AEM page making API calls from Java servelts.
In this article we will go through following process to work in IntelliJ with AEM codebase: 1. Setup our maven AEM archetype 22 project. 2. Learn how to import it into IntelliJ IDE. 3. Using filevault to push and pull code from crx.
AEM Maven project setup:
As I am using the archetype 22 to build my maven project, below is the maven command to create a maven project in local directory:
If you get a build failure message at end then please check for the settings.xml file at C:\Users\<user>\.m2 Below is the settings.xml file that I have used to build the project successfully as it contains Adobe Public Profile, you can copy the same at the above location:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
| This is the configuration file for Maven. It can be specified at two levels:
|
| 1. User Level. This settings.xml file provides configuration for a single user,
| and is normally provided in ${user.home}/.m2/settings.xml.
|
| NOTE: This location can be overridden with the CLI option:
|
| -s /path/to/user/settings.xml
|
| 2. Global Level. This settings.xml file provides configuration for all Maven
| users on a machine (assuming they're all using the same Maven
| installation). It's normally provided in
| ${maven.home}/conf/settings.xml.
|
| NOTE: This location can be overridden with the CLI option:
|
| -gs /path/to/global/settings.xml
|
| The sections in this sample file are intended to give you a running start at
| getting the most out of your Maven installation. Where appropriate, the default
| values (values used when the setting is not specified) are provided.
|
|-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ~/.m2/repository
<localRepository>/path/to/local/repo</localRepository>
-->
<!-- interactiveMode
| This will determine whether maven prompts you when it needs input. If set to false,
| maven will use a sensible default value, perhaps based on some other setting, for
| the parameter in question.
|
| Default: true
<interactiveMode>true</interactiveMode>
-->
<!-- offline
| Determines whether maven should attempt to connect to the network when executing a build.
| This will have an effect on artifact downloads, artifact deployment, and others.
|
| Default: false
<offline>false</offline>
-->
<!-- pluginGroups
| This is a list of additional group identifiers that will be searched when resolving plugins by their prefix, i.e.
| when invoking a command line like "mvn prefix:goal". Maven will automatically add the group identifiers
| "org.apache.maven.plugins" and "org.codehaus.mojo" if these are not already contained in the list.
|-->
<pluginGroups>
<!-- pluginGroup
| Specifies a further group identifier to use for plugin lookup.
<pluginGroup>com.your.plugins</pluginGroup>
-->
</pluginGroups>
<!-- proxies
| This is a list of proxies which can be used on this machine to connect to the network.
| Unless otherwise specified (by system property or command-line switch), the first proxy
| specification in this list marked as active will be used.
|-->
<proxies>
<!-- proxy
| Specification for one proxy, to be used in connecting to the network.
|
<proxy>
<id>optional</id>
<active>true</active>
<protocol>http</protocol>
<username>proxyuser</username>
<password>proxypass</password>
<host>proxy.host.net</host>
<port>80</port>
<nonProxyHosts>local.net|some.host.com</nonProxyHosts>
</proxy>
-->
</proxies>
<!-- servers
| This is a list of authentication profiles, keyed by the server-id used within the system.
| Authentication profiles can be used whenever maven must make a connection to a remote server.
|-->
<servers>
<!-- server
| Specifies the authentication information to use when connecting to a particular server, identified by
| a unique name within the system (referred to by the 'id' attribute below).
|
| NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
| used together.
|
<server>
<id>deploymentRepo</id>
<username>repouser</username>
<password>repopwd</password>
</server>
-->
<!-- Another sample, using keys to authenticate.
<server>
<id>siteServer</id>
<privateKey>/path/to/private/key</privateKey>
<passphrase>optional; leave empty if not used.</passphrase>
</server>
-->
</servers>
<!-- mirrors
| This is a list of mirrors to be used in downloading artifacts from remote repositories.
|
| It works like this: a POM may declare a repository to use in resolving certain artifacts.
| However, this repository may have problems with heavy traffic at times, so people have mirrored
| it to several places.
|
| That repository definition will have a unique id, so we can create a mirror reference for that
| repository, to be used as an alternate download site. The mirror site will be the preferred
| server for that repository.
|-->
<mirrors>
<!-- mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|
<mirror>
<id>mirrorId</id>
<mirrorOf>repositoryId</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://my.repository.com/repo/path</url>
</mirror>
-->
</mirrors>
<!-- profiles
| This is a list of profiles which can be activated in a variety of ways, and which can modify
| the build process. Profiles provided in the settings.xml are intended to provide local machine-
| specific paths and repository locations which allow the build to work in the local environment.
|
| For example, if you have an integration testing plugin - like cactus - that needs to know where
| your Tomcat instance is installed, you can provide a variable here such that the variable is
| dereferenced during the build process to configure the cactus plugin.
|
| As noted above, profiles can be activated in a variety of ways. One way - the activeProfiles
| section of this document (settings.xml) - will be discussed later. Another way essentially
| relies on the detection of a system property, either matching a particular value for the property,
| or merely testing its existence. Profiles can also be activated by JDK version prefix, where a
| value of '1.4' might activate a profile when the build is executed on a JDK version of '1.4.2_07'.
| Finally, the list of active profiles can be specified directly from the command line.
|
| NOTE: For profiles defined in the settings.xml, you are restricted to specifying only artifact
| repositories, plugin repositories, and free-form properties to be used as configuration
| variables for plugins in the POM.
|
|-->
<profiles>
<!-- profile
| Specifies a set of introductions to the build process, to be activated using one or more of the
| mechanisms described above. For inheritance purposes, and to activate profiles via <activatedProfiles/>
| or the command line, profiles have to have an ID that is unique.
|
| An encouraged best practice for profile identification is to use a consistent naming convention
| for profiles, such as 'env-dev', 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc.
| This will make it more intuitive to understand what the set of introduced profiles is attempting
| to accomplish, particularly when you only have a list of profile id's for debug.
|
| This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.
<profile>
<id>jdk-1.4</id>
<activation>
<jdk>1.4</jdk>
</activation>
<repositories>
<repository>
<id>jdk14</id>
<name>Repository for JDK 1.4 builds</name>
<url>http://www.myhost.com/maven/jdk14</url>
<layout>default</layout>
<snapshotPolicy>always</snapshotPolicy>
</repository>
</repositories>
</profile>
-->
<!--
| Here is another profile, activated by the system property 'target-env' with a value of 'dev',
| which provides a specific path to the Tomcat instance. To use this, your plugin configuration
| might hypothetically look like:
|
| ...
| <plugin>
| <groupId>org.myco.myplugins</groupId>
| <artifactId>myplugin</artifactId>
|
| <configuration>
| <tomcatLocation>${tomcatPath}</tomcatLocation>
| </configuration>
| </plugin>
| ...
|
| NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
| anything, you could just leave off the <value/> inside the activation-property.
|
<profile>
<id>env-dev</id>
<activation>
<property>
<name>target-env</name>
<value>dev</value>
</property>
</activation>
<properties>
<tomcatPath>/path/to/tomcat/instance</tomcatPath>
</properties>
</profile>
-->
<!-- ====================================================== -->
<!-- A D O B E P U B L I C P R O F I L E -->
<!-- ====================================================== -->
<profile>
<id>adobe-public</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<releaseRepository-Id>adobe-public-releases</releaseRepository-Id>
<releaseRepository-Name>Adobe Public Releases</releaseRepository-Name>
<releaseRepository-URL>https://repo.adobe.com/nexus/content/groups/public</releaseRepository-URL>
</properties>
<repositories>
<repository>
<id>adobe-public-releases</id>
<name>Adobe Public Repository</name>
<url>https://repo.adobe.com/nexus/content/groups/public</url>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>adobe-public-releases</id>
<name>Adobe Public Repository</name>
<url>https://repo.adobe.com/nexus/content/groups/public</url>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>adobe-public</activeProfile>
</activeProfiles>
</settings>
Once the build is successful, you will get the project structure created in your local directory.
Local project structure
2. Import project to IntelliJ IDE
Import ProjectSelect the root POM fileUse the default settings.
After clicking Next and Finish, project will be imported in the IntelliJ IDE
Now we will deploy this code to out local AEM instance and learn how to pull and push the code from crx
3. Filevault plugin:
We need to first install the IntellIj plugin by going to the plugin section in the settings (Ctrl + Alt + S )
After installation we should have a filevault zip which we will configure in the IntelliVaul. This used to be earlier avialable in AEM 6.2 at C:\Users\<user>\Documents\AEM 6.2\crx-quickstart\opt\filevault.
Download filevault from this link filevault.zip You can reach out to me if you face any issue while downloading fielvault.zip plugin.
This file needs to be unzipped and placed in the same folder in AEM 6.5 and path (..\crx-quickstart\opt\filevault\vault-cli-3.1.6\bin) should be provided as part of configuration in IntelliJ.
Now as the code is deployed to the AEM localhost, we can create number of components and we can import them to our local repo in just few clicks using this vault tool.
I have created a testVault component as part of this demo, now I will try to pull this component to my local repo.
Node in AEM server
Now we will go to the parent node of the node that we want to sync and right and go the IntelliVault option and pull from CRX.
Now we can see the updated component in our local repo. This solves the problem of updating the local code with the code on the AEM server and now updated files are ready to push.
Before starting with the setup of the Elastic search as a service we should have our account setup in https://app.swiftype.com/home. We can use Google id or can create with any other email id as well after email confirmation.
Advantage of using Elastic Search as Service is that it provides the dashboard to handle all kind of requests and uses most of the latest features on swiftype.
To start with, it provides the data indexing using API call and Crawler. Let’s try to understand each of these one by one:
1. API Approach
Once the account is created on swiftype, go to the create engine option and select index data using API and create a engine. Engine Name : bookstore (It’s the engine name) DocumentType Name : book (This can be treated as different tabs or columns)
After creating Engine and document type.
We can create the Engine and DocumentType using the POST requests from Postman with following commands as well:
Go to the create engine options and create new engine and provide the public website URL to index the data from that site: After we provide the URL it identifies the available sitemap to index the data and once you start, it index all the data and can be viewed under content option in dashboard.
Once whole crawling is completed, we can integrate this crawler with our AEM site by placing the script we get from the Install search option on this dashboard. It can be placed in the body tag of the page.html
Example of the code snipped for integration of Elastic crawler in AEM:
AEM Queries & Solutions 