How has Generative AI affected security and compliance?

Generative AI is reshaping industries at an incredible pace. Tools for image creation, chatbots, and code generation are driving innovation and pushing productivity to new heights. According to G2's recent “State of Software” report, demand for these AI solutions is surging across industries. But alongside the excitement comes a new wave of challenges in governance, risk, and compliance (GRC).

Are businesses ready to harness the full potential of generative AI while avoiding legal, security, and reputational pitfalls? Let's dive into the key risks and solutions for these three high-growth areas of generative AI.

Read now: G2's State of Software Report: Scrut ranked #3 in GRC Momentum

1. Image generation: Governance and compliance minefields?

AI-driven image generation tools like DALL-E and Midjourney create rapid, low-cost visual content for marketing and media. But this freedom could create real risks.

Security issues

Image generation software can be impacted by data poisoning, where content publishers subtly alter digital images to disrupt AI training and processing. These “poisoned” images can cause AI models to output flawed results, such as altering the intended object or adding unintended distortions.

Strategies to address this security risk include:

• Employee awareness: Train teams to recognize data poisoning tactics and the potential risks they pose to AI model and output integrity.
• Limit image scraping and browsing to authorized sites: Especially when AI tools can access the internet, ensuring there is a predefined list of approved destinations can reduce the risk of data poisoning from unknown sources.

Litigation and reputation risks

Image generation tools easily produce content that can draw claims of intellectual property (IP) infringement or lead to reputational damage. While the applicability of current law to generative AI is not entirely clear and will continue to unfold, companies can still proactively manage risk.

How to minimize risk:

• Set clear guidelines: Define acceptable use of AI-generated images, particularly for sensitive branding purposes.
• Control access: Limit tool usage to trained staff and establish permissions for generating and using content.
• Understand legal risk: Work with legal counsel to set a governance framework aligned with your business's risk appetite.

Watch now: ResponsibleAI - Beyond Innovation, into Accountability

Regulatory and compliance demands

As regulations catch up with technology, governments are implementing laws around AI-generated images. The European Union's AI Act, for example, will require companies to disclose when an image has been AI-generated. Failing to comply could lead to fines.

Staying on top of legal requirements and ensuring compliance requires a comprehensive strategy that covers your bases in every jurisdiction.

Read also: Seven focus areas to navigate the EU AI Act

2. AI coding copilots

AI code generation tools like GitHub Copilot let developers code faster and with less effort. However, automating software development presents unique security and IP risks organizations need to consider carefully.

Intellectual property ownership issues

AI code generators are often trained on public sources, including open-source projects with a variety of different licensing terms. Although ongoing litigation will determine the legality of such training, companies can now take practical steps to mitigate risk.

Essential governance steps here include:

• Using provider indemnifications: Explore indemnity clauses from vendors like Microsoft or Google, which may cover certain IP disputes.
• Filtering suggestions: Configure tools to block AI-generated code that exactly matches the public code.
• Documenting everything: Ensure developers log AI-generated code to provide traceability in case of disputes.

Security vulnerabilities

AI copilots can also create vulnerable code that slips through security screenings, exposing applications to cyber threats. According to one paper from late 2021, up to 40% of Copilot's recommendations included security vulnerabilities. A more recent study implies those using AI coding assistants write less secure code but are more confident about security.

Strategies to manage AI coding copilot risk include:

• Enforcing code reviews: Ensure at least two human checks of all AI-generated code to catch vulnerabilities or quality issues.
• Be aware of hallucination-enabled typosquatting: Some third-party libraries are consistently hallucinated (i.e., made up) by generative AI tools. Unfortunately, hackers could learn of these and create malicious packages matching their names. This could cause the malicious code to be accidentally integrated into applications, potentially leading to a data breach.

Read also: AI Hallucination: When AI experiments go wrong

3. Chatbots: Risks and controls

AI chatbots have revolutionized customer service. They work around the clock, respond instantly, and save costs. However, they also introduce new privacy and data handling issues, especially when interacting with personal data.

Governance risks

Since chatbots can process sensitive customer data, they need rigorous governance. Clear policies around information handling, customer consent, and transparency are essential. A lapse here can expose companies to breaches, penalties, and reputation damage.

Core data governance practices are:

• Labeling, storage, and retention procedures: Specify how customer data collected by chatbots should be processed, stored, and deleted.
• Transparency with users: Ensure users know when they're speaking to AI to meet ethical and regulatory (specifically the EU AI Act's) demands.
• Assign accountability: Create specific teams responsible for managing chatbot compliance and user data. Just like a service account can't be left without someone in charge, the same goes for AI systems.

Read also: Exploring AI use cases in governance, risk, and compliance

Privacy challenges

Chatbots often interact with private customer data, posing a unique risk. If compromised, chatbots can lead to data leaks. Risk reduction techniques include:

• Compressing the risk surface: Structure chatbots to minimize sensitive data processing and retention to what is absolutely necessary.
• Keeping a neutral security policy: Never rely on chatbots themselves to decide on what data a user is authorized to see. Use traditional authentication mechanisms rather than system prompts.
• Continuously monitoring: Use anomaly detection to catch strange chatbot behavior that may indicate a prompt injection or denial of service attack.

Managing generative AI risk with Scrut Automation

Image generators, chatbots, and coding copilots can drive huge productivity improvements. But with these gains come pressing GRC issues. Companies looking to exploit the potential of generative AI without exposing themselves to undue risk should take a proactive stance.

Scrut Automation offers a comprehensive solution to manage these challenges. With tools for managing vendors, ensuring the completion of security awareness training, and establishing policies and procedures, Scrut lets companies confidently explore the benefits of AI without compromising on governance or security.

Ready to learn more? Book a demo now!